Copula-based system and method for management of manufacturing test and product specification throughout the product lifecycle for electronic systems or integrated circuits

ABSTRACT

A method is provided for determining specifications that meet electronic system or integrated circuit product requirements at all stages of the product lifecycle. Early in the product lifecycle design features must be specified. Later in the lifecycle datasheet specifications must be determined and published to customers, and test specifications in manufacturing must be determined. The method includes acquiring data from a test vehicle, fitting the data to a copula-based statistical model using an appropriately programmed computer, and using the statistical model to compute producer- and customer-oriented figures of merit of a product, different from the test vehicle, using the appropriately programmed computer. Different size, fault tolerance schemes, test coverage, end-use (datasheet), and test condition specifications of the product may be modeled. The statistical model is a copula-based and so can take into account dependency among attributes of the product. The copula-based model has features, which enable significant computational efficiencies.

FIELD OF THE INVENTION

The present invention relates generally to the design and manufacturingtest of electronic systems or integrated circuits. More particularly,the invention relates to a system and method that supportsdecision-making in the design and manufacture of electronic systems orintegrated circuit components throughout the product lifecycle.

BACKGROUND OF THE INVENTION

Design and manufacturing test of electronic systems requireconsideration of the effect of variation of individual components on thevariability of the system. Similarly, design and manufacturing test ofintegrated circuit components require consideration of the effect ofvariation of individual circuit modules on the variation of theintegrated circuit itself. Characterizing and modeling this variationsupport decision-making in the design and manufacturing test of bothelectronic systems and integrated circuit components.

Test in the manufacture of electronic systems or integrated circuitcomponents requires specification of test conditions such astemperatures, voltages, frequencies and parametric test limits, as wellas end-use specifications. End-use specifications are given in adatasheet specification document used by designers of systems employingthe electronic (sub-) system or integrated circuit component beingmanufactured. Test specifications and datasheet specifications are setto optimize yield, yet meet quality and reliability requirements, and sohave an important revenue and brand image impact. Each unit tested ischaracterized by many parametric attributes such as power and delay forelectronic systems, or I_(sb), F_(max), bit refresh time, reliabilitylifetimes, etc., for integrated circuit components, all as functions ofenvironmental conditions such as temperature, voltage and frequency.These parametric attributes are dependent (correlated) to variousdegrees in the population of manufactured units. A traditional method ofoptimizing the test manufacturing flow is to characterize a sufficientlylarge sample of units of a specific product by measuring, but notscreening, the multiple parametric attributes over a range oftemperatures, voltages, and frequencies corresponding to possible Testand Use conditions of a future product. Test set points and limits arethen found by filtering the data and computing figures of merit (FOMs)such as yield loss, overkill, and end-use fail fraction so thatmanufacturing cost, and quality and reliability targets are met. What isneeded is a way to do this earlier in the product lifecycle by buildinga statistical model of a product from test vehicle data and then usingthe model to scale the model to the specific die area, bit count, faulttolerance scheme, etc. of a future product. The same statistical modelmay also be used later in the product lifecycle to decide on end-usespecifications to be published to system designers using the component,and even later in the product lifecycle to optimize the testspecification in manufacturing test. A test vehicle is an electronicsubsystem or integrated circuit device specifically designed tofacilitate data acquisition needed to build the statistical model. Thestatistical model must handle multi-variate dependency, and be scalablefrom the conditions of the test vehicle to the hypothetical design andmanufacturing specifications of a future product.

SUMMARY OF THE INVENTION

To address the needs in the art, a method implemented by anappropriately programmed computer for determining specifications thatmeet electronic system or integrated circuit product requirements isprovided. The method includes acquiring data from a test vehicle,fitting the data to a copula-based statistical model using anappropriately programmed computer, and using the copula-basedstatistical model to compute figures of merit of a future electronicsystem or integrated circuit product, different from the test vehicle,using the appropriately programmed computer. Test vehicles and productshave multiple dependent (correlated) attributes, which are comprehendedby the copula-based statistical model used to fit the test vehicle data.The computed figures of merit of the product are compared with targetvalues of the figures of merit to determine design and manufacturingspecifications of the product.

According to one embodiment of the invention, the test vehicle dataincludes values of attributes for each member of a population of thetest vehicles manufactured by an integrated circuit manufacturingprocess measured as a function of environmental conditions. In oneaspect, the environmental conditions can include temperature, voltage,or frequency.

In another embodiment of the invention, the copula-based statisticalmodel describes a dependency structure of the data.

According to a further embodiment of the invention, the copula-basedstatistical model includes a copula and marginal distribution functionsthat describe a statistical distribution of each attribute of the data,where the copula and the marginal distribution functions embody adependency on environmental conditions. In one aspect, the environmentalconditions can include temperature, voltage, or frequency. In a furtheraspect, if the environmental dependencies are such that the copula doesnot depend on environmental conditions, and the marginal distributionfunctions depend on all environmental conditions through a singlecharacteristic parameter, then useful flexibility in establishing setpoints in test specifications and datasheet specifications obtains. Inanother aspect, the copula is a geometrical copula that enablesnon-reject Monte-Carlo synthesis of synthetic data used to compute thefigures of merit. According to another aspect, the copula of thestatistical model has a tail dependency structure characteristic to thephysics of both the test vehicle and the product. In yet another aspect,the copula is used to generate synthetic Monte-Carlo samples ofinstances of units with multiple attribute values, where the instancesof units correspond to a censored sample of a population of the product,and where attribute values are compared to the test specifications andthe datasheet specifications to determine a pass or fail status of eachinstance, and where the figures of merit are determined by countinginstances of the pass and fail status.

According to another embodiment of the invention, the specificationsinclude design, test and datasheet specifications.

According to another embodiment, the invention further includesdetermining figures of merit and their statistical confidence limits byefficient non-reject Monte-Carlo synthesis of censored synthetic productdata for any experimental design, not only the experimental design whichproduced the test vehicle data, using the appropriately programmedcomputer. The efficient non-reject Monte-Carlo synthesis is enabled bychoosing a geometrical copula to represent the dependency structure ofthe test vehicle data in the statistical model.

According to another embodiment, the invention includes determiningfigures of merit and their statistical confidence limits for theexperimental design which produced the test vehicle data, by usingresampling methods, of which one embodiment is the Bootstrap method.

In a further embodiment of the invention, the fitting includes fittingindividual marginal attribute distribution models and the copula for areference test coverage model, where the fitting of the individualmarginal attribute distribution models and the copula may be done in anyorder. A test coverage model specifies the degree of imperfection in themanufacturing test screen for the attribute it is directly measuring.

According to yet another embodiment of the invention, the acquisition ofthe data using the test vehicle includes measuring attributes separatelyon sub-elements, called “modules”, of the test vehicle, which are alsomodules of the product, or are similar to modules of the product.

In one embodiment of the invention, the acquisition of the data usingthe test vehicle includes a test program that disables all faulttolerance mechanisms in the test program and the test vehicle.

In a further embodiment of the invention, the acquisition of the datausing the test vehicle includes an experimental design having conditionsspanning possible datasheet specifications and test specifications of aproduct.

According to another embodiment, the invention further includesdetermining whether the figures of merit of a new product satisfyquality, reliability, and cost requirements, where the new product hasdesign specifications, test specifications and datasheet specificationsthat are different from design specifications and test specifications ofthe test vehicle. In one aspect, the different design specifications anddifferent test specifications include a different test coverage modelfrom a reference test coverage model assumed in determining thestatistical model from test vehicle data. According to another aspect,the different design specifications of the product include a number ofcircuit sub-elements (modules) that is different from the number ofcircuit sub-elements (modules) in the test vehicle. In a further aspect,the different design specifications include fault tolerance mechanismsthat are not enabled or not present in the test vehicle but are enabledin the product. In yet a further aspect, the way in which testspecifications of the test vehicle and product differ include a testprogram for the test vehicle specifically designed to acquire data tobuild the statistical model.

According to another embodiment of the invention, an analytical form ofthe statistical model is used by an appropriately programmed computer toenable deterministic calculation of figures of merit. In one aspect, thedeterministic calculation of figures of merit enables efficientcalculation of variation of figures of merit as part of characterizationof the design of experiment used to obtain test vehicle data, andextract model parameters of the statistical model there from. In anotheraspect, the deterministic calculation of figures of merit makes thestatistical model useful as a component of larger models, which imposeconstraints beyond the targets for figures of merit described in thisinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows 60 retention times measured in 5 groups of 12 for each bit,according to one embodiment of the invention.

FIG. 2 shows how i_(min) and i_(max) were extracted for a single bitfrom the retention time pass/fail record spanning 5 loops of 12retention time measurements, according to one embodiment of theinvention.

FIG. 3 shows data for several bit failures (rows) copied from an Excelworkbook recording 32843 bit failures, according to one embodiment ofthe invention.

FIGS. 4 a-4 c show Venn diagrams of the categories used in Table 3,according to one embodiment of the invention.

FIG. 5 shows counts of bits in SRT and VRT categories as a function ofenvironmental conditions for the nominal skew for retention times lessthan 604 au (arbitrary units of time). Each bar is sampled from 48750000bits, according to one embodiment of the invention.

FIG. 6 shows retention times for the nominal skew at the highestenvironmental condition binned according to each bit's failing retentiontime in Test and in Use, and empirical marginal Test and Usedistributions computed there from, according to one embodiment of theinvention.

FIG. 7 shows a Weibull distribution fitted to the (equal) margins of thenominal skew data in FIG. 6, according to one embodiment of theinvention.

FIG. 8 shows the quality of fit of characteristic retention time to anexponential voltage, Arrhenius temperature, and model in Eq. (2) at 18environmental conditions for the nominal skew, according to oneembodiment of the invention.

FIG. 9 shows Kendall's tau and sample fraction values for the measuredsample of retention time data plotted as a function of environmentalcondition measured by characteristic retention time for the nominalskew, according to one embodiment of the invention.

FIG. 10 shows a geometrical interpretation of the two-dimensionalcumulative distribution function corresponding to perfect correlation,according to one embodiment of the invention.

FIG. 11 shows how the probability mass in any sub-domain of a copula iscomputed from the copula function, according to one embodiment of theinvention.

FIG. 12 shows the relation of the copula and corresponding survivalcopula, according to one embodiment of the invention.

FIG. 13 shows a diagonal stripe drawn across the unit square forming apseudo-copula in the first step of constructing the stripe geometricalcopula, according to one embodiment of the invention.

FIGS. 14 a-14 f show examples of synthesized probability density maps ofthe stripe copula for various values of the parameter d, and variousdegrees of censoring, according to one embodiment of the invention.

FIG. 15 shows a pseudo-copula A(u, v) having a shaded wedge-shapedregion symmetrical about the (0, 0)/(1, 1) diagonal as the first step inconstructing the wedge geometrical copula, according to one embodimentof the invention.

FIGS. 16 a-16 f show examples of synthesized probability density maps ofthe wedge copula for various values of the parameter c and variousdegrees of censoring, according to one embodiment of the invention.

FIGS. 17 a-17 c show plots of the single parameter of the wedge,Gaussian, and stripe copulas, which minimized the sum-of-squares of Eq.(55), and for the wedge and stripe copulas the sub-population sample taucomputed using Eq. (4), as a function of environmental conditionmeasured by characteristic retention time, according to one embodimentof the invention.

FIG. 18 shows that, because most points lie above the diagonal, thewedge copula is the best fit of each of the three types of copula, whichare embodiments of the invention.

FIGS. 19 a-19 b show that, considering the shape of the data in FIG. 6,the best-fit wedge copula (FIG. 19 a) is a better representation of thetail dependence of the data than the best-fit stripe copula (FIG. 19 b),comparing two embodiments of the invention.

FIG. 20 shows a schematic drawing of integrated circuit manufacturingprocess (Fab/Assembly) producing units of a product which are tested(Test) per Test Conditions and then go on to be used (Use) per DatasheetSpecifications. The definition of key figures of merit (FOMs) isillustrated, according to one embodiment of the invention.

FIG. 21 shows a schematic drawing of how the Test and Use conditionsdivide the population of manufactured units into categories, accordingto one embodiment of the invention.

FIG. 22 shows a schematic representation of how the Test and Useconditions superimposed on the DRAM bit pseudo-copula's pdf (shaded)divides the bit population into four Use/Test pass/fail categories,according to one embodiment of the invention.

FIGS. 23 a-23 c show how three probability functions used to computeFOMs correspond to shapes in bit category index space enclosingtolerated counts of to memory array bits in various categories for theNo Repair at Test case, according to one embodiment of the invention.

FIGS. 24 a-24 b show the bit category index space volumes enclosingtolerated bit counts in various categories corresponding to Overkill(Mfg) and EUFF for the No Repair at Test case of a memory array,according to one embodiment of the invention.

FIGS. 25 a-25 c show the Passes Test (FIG. 25 a) and Good in Use (FIG.25 b) categories represented as volumes in bit category index space, andthe intersection of these categories (FIG. 25 c), for a memory array inthe Repair at Test case, according to one embodiment of the invention.

FIG. 26 shows an example user interface of an Excel calculator, whichimplements the models in this invention.

FIGS. 27-28 show graphs of figures of merit vs. test retention timesetting, with (FIG. 27), and without (FIG. 28), fault toleranceaccording to one embodiment of the invention.

FIGS. 29 a-29 c show, for the case of no fault tolerance, graphs offigures of merit for test coverage models different from theconservative test coverage model

(FIGS. 29 a and 29 b), and for the case of perfect correlation betweentest and use (FIG. 29 c), according to one embodiment of the invention.

FIGS. 30 a-30 b compare a case for which Test tolerates, but does notrepair, up to two failing bits with a case for which Test tolerates andrepairs up to two failing bits, according to one embodiment of theinvention.

FIG. 31 is a graph showing that if Test tolerates and repairs up to twofailing bits, and arrays in Use can tolerate one failing bit, comparedto FIG. 30 b, in which Use tolerates no failing bits, the EUFF isimproved (reduced), and Overkill becomes the main component of YieldLoss, according to one embodiment of the invention.

FIG. 32 a shows a flow diagram of the Model Extraction part of thestatistical methodology in which Monte-Carlo synthesis generates DataReplicates, according to one embodiment of the invention.

FIG. 32 b shows a flow diagram of the Model Extraction part of thestatistical methodology in which resampling of the data using theBootstrap method generates Data Replicates, according to one embodimentof the invention.

FIG. 33 shows a flow diagram of the Inference part of the statisticalmethodology, according to one embodiment of the invention.

FIG. 34 shows a schematic graph of the method for setting guard bands toaccount for sampling variation due to the Experimental Design andParameter Extraction parts of the Model Extraction method, according toone embodiment of the invention.

FIGS. 35 a-35 b show geometrical constructions on the stripepseudo-copula to derive a Monte-Carlo sampling algorithm for a censoredsub-population of the stripe copula, according to one embodiment of theinvention.

FIG. 36 shows basis vectors for sampling the area of uniform probabilityin the wedge pseudo-copula, A(u, v), according to one embodiment of theinvention.

FIG. 37 shows the integration limits for an integral used to derive anexpression for the subpopulation Kendall's tau of the wedge geometricalcopula, according to one embodiment of the invention.

DETAILED DESCRIPTION

A method of determining design, manufacturing test, and end-use(datasheet) specifications for an electronic system or integratedcircuit product is provided. Since the method applies to both electronicsystems and to integrated circuits it will be convenient to describe themethod in terms of application to only one of these, namely, anintegrated circuit. In the following, the term “test chip” for theintegrated circuit (IC) case would be replaced by “prototype system” (orsimilar) for the system case; “module” (or “bit” for the DRAM example)for the IC case would be replaced by “component” for the system case,and so on.

The method includes acquisition of data from a test vehicle, where thetest vehicle can be a test chip or product different from the productfor which specifications are being determined. Differences between theproduct and the test vehicle may include different test specifications,datasheet specifications, and different design specifications. Examplesof design specifications include the number (e.g. array size) of modules(e.g. bits) and the degree and type of fault tolerance required. Themethod further includes fitting the test vehicle data to a statisticalmodel using an appropriately programmed computer, and using thestatistical model to compute figures of merit using the appropriatelyprogrammed computer. Since the data acquired using a test vehicleincludes measurements of dependent (e.g. correlated) attributes, it isnecessary for the statistical model to describe this dependency. Themethod uses a copula-based statistical model to comprehend thedependency. The copula-based model is used to compute figures of meritand compare them with target values to determine specifications thatmeet requirements of the integrated circuit product. Because ofproperties unique to copulas, such as the decoupling of marginaldistributions from the core dependency structure, the copula-basedstatistical model gives significant advantages in model fitting and inmaking inferences.

The following algorithms and equations of the current invention areimplemented in a software tool using a computer, which allows the userto specify attributes of the product such as number of modules (e.g.array size), fault tolerance characteristics, manufacturing test setpoints and limits, and datasheet specifications of the product. The tooloutputs values of figures of merit such as yield loss, overkill, andend-use fail fraction. The algorithms can also be implemented as plug-insoftware modules in larger software programs such as optimizationengines.

The copula can be characterized independently of marginal distributions.For example, empirical rank statistics such as Kendall's tau provide aview into the underlying dependency structure (the copula), withoutinterference by details of marginal distributions. Further, it is likelythat a population copula that is independent of environmental conditionsmay be found since it contains only rank statistical information, whichis relatively undisturbed by environmentally-induced changing marginaldistributions. If an environmentally independent copula can be found, agreat simplification of the model obtains, and advantages such as“equivalent set points”, explained below may be realized. Additionally,copula models have an advantage over the prior art because differentcopula models may be tried without changing any other part of theanalysis, whereas for multi-normal models the marginal and dependencystructures are entangled.

Copula-based methods have a much greater flexibility than themulti-normal-based methods usually employed to represent the dependencystructure of parametric attributes. Multi-normal statistics are aspecial case of the Gaussian copula described in this invention, andshare the shortcomings of the Gaussian copula. A key shortcoming of theGaussian copula, and therefore of the usual multi-normal model, is thatthese models cannot properly represent correlation that extends deepinto the tail of multi-variate distributions, whereas, according to thecurrent invention, many copulas can do this. The DRAM example belowrequires a deep-tail-dependence copula, which will be shown not to besatisfactorily modeled using the Gaussian copula.

An example case study using DRAM retention time correlation between Testand Use is provided to demonstrate the current invention of acopula-based statistical modeling method which is not limited tointegrated circuit memory arrays, but is also applicable to more complexintegrated circuits. Further, the method described here can be appliedto other dependent attributes measured in the same test socket,different test sockets, and being manifested in end-use (“Use”).Examples include exploiting in a manufacturing environment thecorrelation of measured attributes like I_(sb) to difficult- orexpensive-to-measure attributes like F_(max), power, or reliabilitylifetime. The invention enables quantitative characterization, in termsof well-defined figures of merit, of methods in which measuredattributes are used to used to screen hard-to-measure correlatedattributes.

Aspects unique to the application of copula methods to manufacturingtest and product specification application include: Fitting andefficient synthesis of highly censored data across environmentalconditions; Construction of a new kind of “geometrical” copula; Precisedefinition of figures of merit such as yield loss, overkill andcustomer-perceived fail fraction; Scaling by product size; Faulttolerance. Both model fitting and inference aspects of the copula-basedstatistical modeling method are covered by the invention.

Turning to the example, each memory bit of a dynamic random accessmemory (DRAM) retains its information as stored charge on a capacitor.After the bit has been written to, the charge leaks away so that validdata has a characteristic retention time. To retain the information, thebit must be read and refreshed. DRAM memory cells can have a defect,which causes a few bits to have a variable retention time (VRT), whilemost bits have stable retention times (SRT). The VRT behavior is causedby a silicon lattice vacancy-oxygen complex defect when the defect isembedded in the near surface drain-gate boundary of a DRAM cell. Thedefect can transition reversibly between two states. One of the statesis associated with a leaky capacitor and short retention times. The VRTmechanism causes a soft-error reliability issue in DRAM arrays since aVRT bit may be tested and pass a brief retention time screen while inthe low leakage state, but in extended use, a high leakage state willalmost certainly occur possibly with retention time less than thesystem-specified refresh time. This miscorrelation of retention timebetween test and use is perceived as a soft-error reliability issue,which requires error correction methods to make the DRAM tolerant ofsingle bit errors. This example of the invention shows how to measureand model this miscorrelation in a DRAM case study to establish testspecifications such as set points (temperature, voltage) and limits(retention time), establish datasheet specifications (temperature,voltage, refresh time), and determine sufficient levels of faulttolerance to meet quality and reliability requirements.

Test chips with four identical DRAM arrays on each chip were fabricatedin five skews of a 65 nm process as shown in Table 1. Each of the fourarrays on a chip has 1218750 bits. The test chips were packaged in ballgrid array packages and 10 randomly selected test chips from each of theprocess skews were selected for this example.

TABLE 1 Five process skews were produced for this experiment. Slowerskews have longer retention times. Name Description Nominal Nominalprocess. Slow NMOS Slow, PMOS Slow Fast/Slow NMOS Fast, PMOS SlowSlow/Fast NMOS Slow, PMOS Fast Very Slow NMOS Slow, PMOS Slow, 20%larger than nominal capacitors.

The arrays were tested on a Credence Quartet tester with 145 I/Os andseven power supplies and the temperature was controlled by a SiliconThermal Powercool LB300-i controller. Retention time for each bit wasmeasured at 18 environmental conditions:

-   -   three temperatures: 105° C., 115° C., 125° C. Temperature was        measured by a calibrated sensor on the silicon die.    -   three V_(d)'s: 0.8, 1.0, 1.2 volts. V_(d) is the supply voltage.    -   two V_(p)'s: 0.4, 0.45 volts. V_(p) is the substrate bias.

60 retention times in five groups of 12 were measured for each bit,shown in FIG. 1, and as follows:

-   -   12 retention times (r) were tested, increasing from 60 au to 604        au in steps of 49.5 au: r=10 +i×49.5 au, i=1 to 12, with “pass”        or “fail” determined at each retention time. To obscure        proprietary aspects of the data, and the fitted model, retention        times are given in arbitrary units (au), which are related to        the true retention times by a numerical ratio.    -   This was repeated five times, with each repetition called a        “loop”. The repeated loops were separated by variable durations,        typically many hours.

At each environmental condition the pattern (FIG. 1) of retention timefailures observed for each bit was used to classify a bit as not failing(r>604 au), or as failing by either variable retention time (VRT) or bystable retention time (SRT). If a bit failed on the first reading on anyloop, it was also classified as a zero retention time ZRT bit. At eachenvironmental condition any given failing bit may be classified as SRTor VRT and ZRT or not-ZRT. The classification of a given bit may bedifferent in a different environmental condition.

EXAMPLES

SRT: r index varies by ≦1 within loop, or loop-to-loop (two examples).

-   -   000000011111 000000011111 000000011111 000000011111 000000011111        000000001111 000000001111 000000011111 000000001111 000000011111

VRT: r index varies by ≧2 within loop, or loop-to-loop (two examples).

-   -   000000000111 000011111111 000000001111 000000001111 000000000111        000110111111 000001111111 000111101111 000001111111 000111111111        (This pattern is used in an example below).

SRT/ZRT: r index=0 in any loop. r index varies by ≦1 within loop, orloop-to-loop (two examples).

-   -   111111111111 111111111111 1111111111111 111111111111        111111111111 111111111111 111111111111 111111111111 011111111111        011111111111

VRT/ZRT: r index=0 in any loop, and r index varies by ≧2 within loop, orloop-to-loop (one example).

-   -   111111111111 111111111111 001111111111 111111111111 011111111111

For each bit and each environmental condition, the pass/fail patternswere processed to extract the index of the smallest passing retentiontime, i_(min), and the index of the longest passing retention time,i_(max), i_(max) was found by “AND-ing” all loops for a bit and findingthe index of the first “0” counting from the right. i_(min) was foundfor the same loop data by “OR-ing” all loops and finding the last “0”counting from the left. The method is shown in FIG. 2 for the second VRTexample above. If the first measurement of retention time in any loop isa failure (“1”), then i_(min)=0.

Bits for which i_(min)=0 in any loop were classified as zero retentiontime (ZRT) bits. Bits for which i_(max)−i_(min)≦1 were classified asstable retention time (SRT) bits. The margin of 1 allows for thepossibility of tester variation in case the bit retention time falls ona retention time bin boundary. Bits for which i_(max) −i_(min)≧2 wereclassified as variable retention time (VRT) bits. The reason forrepeating the retention time measurement sequence five times is to givebits plenty of opportunity to show variable retention time behavior.

Provided is a description of the data, where a total of 32843 bitretention time failures were recorded from 48750000 bits in each skewacross 18 environmental conditions (temperature and voltage). The datawere censored because only bits failing with retention times less than604 au were recorded. The same bit could be recorded as a failuremultiple times because, within a skew, the same bit may fail in multipleenvironmental conditions. FIG. 3 shows data for a few failures (rows)copied from the Excel workbook containing all the data. The field names(columns) are defined in Table 2.

TABLE 2 Definitions of data fields used in DRAM data records. GroupField Description Identity skew Process skew code. Values = 4, 8, 9, 10,12. chip Which of 10 chips sampled contains the macro. Values = 1-10.macro Macro which contains PX, PY bit. Values = 0, 1, 2, 3. PX, PY x, ycoordinates of bit. Environmental Vp Value of Vp. Values = 0.4, 0.45 VdValue of Vd. Values = 0.85, 1.0, 1.2 temp Temperature (° C.) Values =105, 115, 125. Results of Test IRetMin Minimum retention time indexderived per FIG. 2. If IRetMin = 0, the bit is a ZRT bit. IRetMaxMaximum retention time index derived per FIG. 2. IRetDelta IRetDelta =IRetMax-IRetMin If IRetMin > 0 and IRetDelta ≦ 1, the bit is a SRT bit.If IRetMin > 0 and IRetDelta > 1, the bit is a VRT bit Loop GroupsPass/Fail record of bit at specified environmental condition.

Provided below is a classification of bits into categories to give anoverview of the nature and significance of the miscorrelation phenomenonto be analyzed by the method of this patent. The data in FIG. 3 wereused to classify the observed failing bits as either SRT or VRT, andeach of these as ZRT or not-ZRT for each environmental condition. Anygiven to bit could be classified differently in any of the 18environmental conditions. A summary of the statistics when each categoryis OR-ed across all environmental conditions is given in Table 3. Thisgives a measure of the fraction of defective bits of various kinds ineach skew. For example, in the nominal skew, 1610 of 48750000 bitsremained as SRT bits across all environmental conditions; although theretention times varied from bit to bit they were always stable throughtime. Also, for this skew, 288 of 48750000 bits were observed as eitherSRT or VRT depending on the environmental condition. And 64 of 48750000bits were observed only as VRT bits in all 18 environmental conditions.Also, 10 of 288+64=352 bits which exhibited VRT behavior were also seenas ZRT bits at some environmental condition, and four of 1610 bits whichexhibited only SRT behavior were seen as ZRT bits in some environmentalcondition.

An important observation from Table 3 is that for the Nominal skew18%=(B+C)/(A+B+C) of the 40 DPPM of bits observed to fail within theenvironmental and retention time span of the experiment exhibited VRTbehavior. Since the environmental and retention time settings spanrealistic Test and Use conditions, the number of SRT/VRT failuresimplies that bit repair or fault tolerance schemes will be necessary inthe design of any practical DRAM array with thousands or millions ofbits. It also shows that the VRT failure mode is a significantcontributor to the soft error failure rates, along with contributionsfrom other sources such as cosmic rays.

TABLE 3 Bit-count and defective parts-per-million (DPPM) statistics ofbit failure modes OR-ed across all environmental conditions. The keysare explained by the Venn diagrams in FIGS. 4a-4c. Skew Description KeyNominal Slow Fast/Slow Slow/Fast Very Slow SRT, but not VRT A 1610 8332092 1042 501 VRT and SRT B 288 124 787 280 83 VRT, but not SRT C 64 24141 73 10 ZRT and VRT D 10 4 10 6 0 ZRT and SRT, but not VRT E 4 2 4 2 0Total Bits N 48750000 48750000 48750000 48750000 48750000 SRT (incl.ZRT) DPPM A/N 33.0 17.1 42.9 21.4 10.3 VRT (incl. ZRT) DPPM (B + C)/N7.2 3.0 19.0 7.2 1.9 ZRT DPPM (D + E)/N 0.29 0.12 0.29 0.16 0.00

The environmental dependence of the SRT and VRT categories is shown inFIG. 5. Within each environmental condition, these categories aremutually exclusive. The strong temperature and voltage dependence isapparent. It will be shown that these data are accurately fitted withexponential voltage dependence, and Arrhenius temperature dependence.

The method of this patent requires extraction of a statistical model forthe actual retention times for each bit, rather than of the coarsecategories (SRT, etc.) into which bits may fall according to theirretention time behavior. In the example experiment, for a given skew andenvironmental condition, every failing bit will have minimum retentiontime of r_(min) and a maximum retention time of r_(max) observed in thecourse of 60 repeated measurements (12 measurements in five loops). Onthe other hand, in Test and Use conditions, the retention time of a bitat Test, r_(test), is measured just once, and the retention time in Use,r_(use), is sampled an indefinitely large number of times as the memoryis used.

It is necessary to associate r_(max) and r_(min) with r_(test) andr_(use) in a way that does not underestimate the failure rate of bits inuse. This association is an example of a “test coverage model”. The mostconservative association from the customer standpoint, predicting thehighest escape rate, is r_(test)=r_(max), and r_(use)=r_(min). Thisassociation is called the “conservative test coverage model”. It isclear that r_(use) should be associated with r_(m), since r_(min) , isthe result of many repeated measurements, mimicking Use. In reality, theprobability that r_(max) will occur in Test depends on the fraction oftime a bit is in the maximum retention time state. Since this fractioncannot be determined from the results of the experiment because theintervals between measurements are not precisely known, it isconservative to assume that Test always measures r_(max). Measurement ofthe fraction of time a bit spends in high and low retention time stateswould provide information allowing this conservative assumption to berelaxed.

Although the model that is ultimately used assumes r_(test)=r_(max), andr_(use)=r_(min) it is more convenient for the initial model extractionto assume that the assignment r_(test)=r_(max)/r_(use)=r_(min), andr_(test)=r_(min)/r_(use)=r_(max) are made with equal probability foreach bit. This assignment is called the “symmetrical test coveragemodel”. Shown below is an analytical way to invoke the conservative testcoverage model that r_(test=r) _(max)/r_(use)=r_(min) when makinginferences from the model extracted assuming the symmetrical testcoverage model. Besides simplifying model extraction, this technique hasthe advantage of enabling exploration of the sensitivity of inferencesto any chosen test coverage model. This might be used to evaluate thereturn on the investment of a more precise time-in-statecharacterization of variable retention times. An example of an empiricalprobability density function for a single skew at the highestenvironmental condition derived by assuming a symmetrical test coveragemodel is shown in FIG. 6.

The next step, shown in the dashed-line areas of FIG. 6, is to extractempirical marginal distributions for Test and Use by summing fail countsin rows, and in columns, and then computing the cumulative fail countand cumulative fraction fail for each. This was done for each of the 5skews at each of the 18 environmental conditions. The marginaldistributions of Test and Use for the DRAM are the same, apart from somesampling noise, because of the symmetrical way in which r_(min), andr_(max) were assigned to r_(use) and r_(test). Notice from the figurethat only a tiny part of the entire sample space near the origin is ofpractical significance—a fraction of only about 34 parts per million(PPM) of the sample space was observed. This is typical of integratedcircuit test correlation data.

Also typical of parametric test data in general is the strong increasein cumulative fraction failing as a function of the test condition,which suggests fitting of the empirical marginal distributions in FIG. 6to a Weibull model distribution. The Weibull distribution is a naturalchoice because, with a shape parameter β>1, it can fit data for whichthe proportional increase in fraction failing per unit increase inretention time is a strongly increasing function, as observed here. FIG.7 is a plot of ln(−ln(1−F)), known as the “Weibit”, versus ln(r). Slopeand intercept of lines fitted to these data give estimates of the shape,β, and scale, α, parameters of the Weibull distribution of retentiontime, r, in

$\begin{matrix}{{{{F(r)} = {1 - {\exp \left\lbrack {- \left( \frac{r}{\alpha} \right)^{\beta}} \right\rbrack}}},{or}}{{Weibit} = {{\ln \left( {- {\ln \left( {1 - F} \right)}} \right)} = {{{\beta ln}(r)} - {\beta \; {\ln (\alpha)}}}}}} & (1)\end{matrix}$

for both Test and Use.

The fitted marginal distributions were simplified by forcing β=2.0 inthe fit of all marginal distributions, as exemplified by the dashed linein FIG. 7. Visual examination of all fitted distributions shows thatthis gives a small underestimate of the retention time at shortretention times. Such an underestimate is conservative from thestandpoint of customer-perceived quality. This analysis was repeated foreach of 18 environmental conditions for each of five skews.

For each skew, this method extracts a value of the natural log of thecharacteristic retention time, lnα, for each of 18 environmentalconditions. The characteristic retention time, α, accurately fits anexponential voltage dependence and Arrhenius temperature dependence:

$\begin{matrix}{{\ln \; \alpha} = {{\ln \; \alpha_{0}} + {a\left( {V_{p} - V_{p\; 0}} \right)} + {b\left( {V_{d} - V_{d\; 0}} \right)} + {\frac{Q}{k_{B}}\left( {\frac{1}{T} - \frac{1}{T_{0}}} \right)}}} & (2)\end{matrix}$

where the subscript 0 indicates a reference environmental condition. Thereference condition has been chosen arbitrarily as the maximum stresscondition in the experiment. lnα₀ is the natural logarithm of thecharacteristic retention time in au at this condition. V_(d) is thesupply voltage, V_(p) is the substrate bias voltage, and T (° K) is thetemperature. The quality of the fit is shown in FIG. 8. It is known thatthe activation energy of retention time is between 0.6 eV and 1 eV forSRT bits and VRT bits in the low-leakage state and about 0.2 eV for VRTbits in the high leakage state. In this example the fitted value of Q isabout 0.6 eV (Table 4). This is expected since most observed bits are inthe low leakage state. The low activation energy of VRT bits in the highleakage state could lead to under-estimates of customer risk if themodel is extrapolated beyond the range of the data on the lowtemperature side. However, the data spanned the test conditions and useconditions so extrapolation is not needed.

Regarding the equivalent set point method, environmental conditionsenter the statistical model only through lnα, and through possibleenvironmental dependence of the copula. If, in addition, the fittedcopula model is environmentally independent, as will be seen for thebest fit to the DRAM example (wedge copula), then test specifications ordatasheet specifications which have a given value of r/α arestatistically equivalent and so give the same figures of merit. In Test,this is useful because temperatures cannot be quickly changed from testto test in an integrated test program, whereas voltages and retentiontime settings can be. So rapid voltage and/or retention time settingchanges can be used instead of equivalent slow temperature settingchanges. In Use, equivalence of datasheet specifications with a givenvalue of r/α gives flexibility since temperature or voltage supplyspecifications may be constrained by other components in a system. Theset point flexibility enabled by environmental independence of thecopula, and marginal distribution dependence only on r/α, is called theequivalent set point method.

Kendall's tau is a statistic used in the invention to measure thesimilarity of the rank orderings between a pair of attributes measuredon a population of units. Its value ranges from 1 when the ranks are thesame, through 0 when there is no relationship between ranks, to −1 whenthe ranks are opposite. Suppose that {(x₁, y₁), (x₂, y₂), . . . (x_(n),y_(n))} denotes a random sample of n observations sampled from apopulation 2-vector of continuous random variables, (X, Y). Any pair ofobservations, i and j, may be labeled “concordant”, or “discordant”,according to whether the sense of the inequality between x_(i) and x_(j)is the same or the opposite to the inequality between y_(i) and y_(j).There are a total of n(n−1)/2 pairs of observations, of which c areconcordant and d are discordant. Kendall's tau for the random sample isdefined as

$\begin{matrix}{\tau^{\prime} = {\frac{c - d}{c + d} = \frac{2\left( {c - d} \right)}{n\left( {n - 1} \right)}}} & (3)\end{matrix}$

where the prime indicates a sample estimate of tau. The definition ofthe sample tau in Eq. (3) assumes that the sampled values are known witharbitrary numerical resolution so that they can be ranked with no ties.

Test data, such as FIG. 6, has limited resolution and is “binned” sothat ties in x's and y's occur. The definition, Eq. (3), has a knownextension to take ties into account. The sample tau for data with tiesis

$\begin{matrix}{\tau^{\prime} = \frac{c - d}{\sqrt{{\frac{1}{2}{n\left( {n - 1} \right)}} - U}\sqrt{{\frac{1}{2}{n\left( {n - 1} \right)}} - V}}} & (4)\end{matrix}$

where now any pairs that are tied in x or y are not counted in either cor d, and where

$\begin{matrix}{{U = {\frac{1}{2}{\sum{u\left( {u - 1} \right)}}}},{V = {\frac{1}{2}{\sum{{v\left( {v - 1} \right)}.}}}}} & (5)\end{matrix}$

The sum in Eq. (5) is over all sets of tied x-values, and u is thenumber of tied x values in each set. V is defined in the same way, butfor y-values. Code to implement (4) is known.

For each environmental condition of each skew, Eq. (4) was used tocompute Kendall's tau for the measured sample of retention time data.These values are plotted as a function of environmental condition forthe nominal skew in FIG. 9, along with the fraction of the population of48750000 bits tested. Note that the sample tau is independent ofenvironmental condition (lnα) and the degree of censoring (samplefraction). This was true for all skews, not only the nominal skew shownin FIG. 9. Moreover, since Table 4 shows the average value of tau acrossenvironmental conditions for each skew, it is clear that the value oftau does not vary greatly from skew to skew. This key result isconsistent with an environmental and skew independence of the underlyingdependency structure of the DRAM VRT behavior embodied in the copula.

TABLE 4 Parameters of extracted marginal and dependence models. Sampletau extracted from data via Eq. (4), averaged across environmentalconditions, is also given. Skew Nominal Slow Fast/Slow Slow/Fast VerySlow Margin 2.0 2.0 2.0 2.0 2.0 ln[₀(au)] 11.57 11.93 11.39 11.72 12.15a₀ −5.79 −4.00 −5.71 −5.55 −4.52 b₀ −1.55 −1.09 −1.47 −1.87 −1.23 Q₀(eV) 0.605 0.572 0.658 0.566 0.630 V_(p0) 0.45 0.45 0.45 0.45 0.45V_(d0) 1.2 1.2 1.2 1.2 1.2 T₀ 125 125 125 125 125 Dependence SampleTau0.828 0.828 0.769 0.802 0.822 Wedge Copula c 1.142 1.140 1.185 1.1741.144 Gaussian Copula (1-) × 0.695 0.646 1.250 1.026 0.634 1E3

Turning now to modeling the dependence observed in the example datasummarized in Table 4 using a copula-based statistical model, start byobserving that FIG. 6 is an empirical sampling of a bi-variateprobability density function (pdf), which here is taken to be h(x, y).In this pdf, for convenience, r_(use)/α is denoted by x, and r_(test)/αis denoted by y. The off-diagonal cells in FIG. 6 are populated bycounts of units for which Use and Test measurements of retention timeare miscorrelated. The marginal cumulative distributions for Test andfor Use are also shown in FIG. 6 by the dashed-line areas. Separation ofmodeling of marginal distributions from modeling the dependencystructure of the data is a key aspect and major benefit of the currentinvention.

For perfect correlation the pdf sampled in FIG. 6 would be a line ofprobability density running up the diagonal

h(x, y)=f(x)δ(x−y)   (6)

where δ(·) is Dirac's delta function, and f(x) is the pdf of themarginal distributions (both equal for the DRAM case study),

$\begin{matrix}{{f(x)} = \frac{{F(x)}}{x}} & (7)\end{matrix}$

where F(x) is the cumulative density function (cdf) of the marginaldistributions.

Now, consider the corresponding two-dimensional cdf:

$\begin{matrix}\begin{matrix}{{H\left( {x,y} \right)} = {\int_{0}^{x}{{x^{\prime}}{\int_{0}^{y}{{y^{\prime}}{h\left( {x^{\prime},y^{\prime}} \right)}}}}}} \\{= {\int_{0}^{x}{{x^{\prime}}{\int_{0}^{y}{{y^{\prime}}{f\left( x^{\prime} \right)}{\delta \left( {x^{\prime} - y^{\prime}} \right)}}}}}} \\{= \begin{Bmatrix}\begin{matrix}{{\int_{0}^{x}{{{f\left( x^{\prime} \right)}\left\lbrack {\int_{0}^{y}{{\delta \left( {x^{\prime} - y^{\prime}} \right)}{y^{\prime}}}} \right\rbrack}{x^{\prime}}}} =} \\{{\int_{0}^{x}{{f\left( x^{\prime} \right)} \times 1 \times {x^{\prime}}}} = {F(x)}}\end{matrix} & {y \geq x} \\\begin{matrix}{{\int_{0}^{y}{\left\lbrack {\int_{0}^{x}{{f\left( x^{\prime} \right)}{\delta \left( {x^{\prime} - y^{\prime}} \right)}{x^{\prime}}}} \right\rbrack {y^{\prime}}}} =} \\{{\int_{0}^{y}{{f\left( y^{\prime} \right)}{y^{\prime}}}} = {F(y)}}\end{matrix} & {x \geq y}\end{Bmatrix}} \\{= {\min \left\lbrack {{F(x)},{F(y)}} \right\rbrack}}\end{matrix} & (8)\end{matrix}$

where, for y≧x. [·]=1 because x′ ∈[0, y], and for x≧y. [·]=f(y′) becausey′ ∈[0, x. A geometrical interpretation of Eq. (8) is shown in FIG. 10,where for perfect correlation, the probability density is a deltafunction on the diagonal. The two-dimensional cdf is the probabilitymass enclosed by a rectangle with one corner pinned at (0, 0).

On the other hand, for perfect independence, elementary probabilitytheory gives the two dimensional cdf as the product of the marginal cdfs

H(x, y)=F(x)F(y).   (9)

Generalizing, if the marginal cdfs are different, for perfectcorrelation

H(x, y)=min[F(x),G(y)],   (10)

and for perfect independence

H(x, y)=F(x)G(y).   (11)

A copula is the multi-dimensional cdf written as a function of themarginal cdfs, rather than the marginal variables,

H(x, y)=C[F(x),G(y)]  (12)

or

C(u, v)=H(F ⁻¹(u),G ⁻¹(v)),   (13)

so Eqs. (10) and (11) are special cases of Eq. (12) where C is one ofthe first two of the following special copulas

$\begin{matrix}{{C\left( {u,v} \right)} = \left\{ \begin{matrix}{\min \left\lbrack {u,v} \right\rbrack} & {{M\left( {u,v} \right)}\mspace{14mu} {Perfect}\mspace{14mu} {correlation}} \\{uv} & {{\Pi \left( {u,v} \right)}\mspace{14mu} {Independence}} \\{\max \left\lbrack {{u + v - 1},0} \right\rbrack} & {{W\left( {u,v} \right)}\mspace{14mu} {Perfect}\mspace{14mu} {anti}\text{-}{{correlation}.}}\end{matrix} \right.} & (14)\end{matrix}$

All possible multi-dimensional cdfs have copulas which are bounded aboveby M, the Frechet Upper Bound, and below by W, the Frechet Lower Bound.Π and all other copulas lie between these limiting functions.

Copulas have four defining properties. A 2-dimensional copula is a cdfwith range [0, 1] on domain [0, 1]² (the unit square) which has thefollowing properties:

-   -   1. Grounded.

C(u, 0)=0=C(0, v).   (15)

-   -   2. Normalized.

C(1, 1)=1.   (16)

-   -   3. Uniform marginal distributions.

C(u, 1)=u and C(1, v) =v.   (17)

-   -   4. 2-increasing. For every u₁, u₂, v₁, v₂ in [0, 1] such that        u₁≦u₂ and v₁≦v₂

C(u, v)−C(u, v)−C(u, v)+C(u, v)≧0.   (18)

The expression on the left side of Eq. (18) is the probability mass inthe region defined by u₁, u₂, v₁, v₂ in [0, 1] such that u₁≦u₂ andv₁≦v₂. The method of computing probability mass in a sub-area of [0, 1]²shown in FIG. 11 is used extensively in the description of thisinvention.

These concepts can be extended in several known ways. One way is toconsider more than two independent variables. Another way is to relaxone or more of the four defining properties of a copula. A pseudo-copularelaxes condition 3, Eq. (17). For a pseudo-copula, condition 3 becomes

C(u, 1)=F(u) and C(1, v)=G(v),   (19)

where F has the properties of a one-dimensional cdf, that is F(0)=0,F(1)=1, and F(u) is a monotonically increasing function and similarlyfor G. In the current invention a copula is fitted to test vehicle data,and then a pseudo-copula is constructed to make model inferences. Apseudo-copula is also the starting point for definition of a class of“geometrical” copulas described in this invention.

Another known extension of the copula concept used here is the survivalcopula, illustrated in FIG. 12. A 2-dimensional copula is the fractionof the population failing by both marginal cdfs (that is, u<1 and v<1)as a function of the marginal fail cdfs, u and v. The correspondingsurvival copula is the fraction of the population surviving by bothmarginal cdfs (that is, u≧1 and v≧1) as a function of the marginalsurvival cdfs ū=1−u and v=1−v. Using Eq. (18), from FIG. 12 it isapparent that the survival copula is related to the copula by

$\begin{matrix}\begin{matrix}{{S\left( {\overset{\_}{u},\overset{\_}{v}} \right)} = {{C\left( {1,1} \right)} - {C\left( {u,1} \right)} - {C\left( {1,v} \right)} + {C\left( {u,v} \right)}}} \\{= {1 - {C\left( {{1 - \overset{\_}{u}},1} \right)} - {C\left( {1,{1 - \overset{\_}{v}}} \right)} + {C\left( {{1 - \overset{\_}{u}},{1 - \overset{\_}{v}}} \right)}}} \\{= {\overset{\_}{u} + \overset{\_}{v} - 1 + {C\left( {{1 - \overset{\_}{u}},{1 - \overset{\_}{v}}} \right)}}}\end{matrix} & (20)\end{matrix}$

where the second equality applies to copulas and pseudo-copulas, and thefinal equality applies to copulas.

The foundation of the copula method is Sklar's theorem, which statesthat the decomposition of a given multidimensional cdf H into itsmarginal distributions and the function C in Eq. (12) is unique. Thatis, there is only one function C, which satisfies Eq. (12) for a givenH. Moreover, it has been shown that the copula corresponding to a jointcdf of statistical variables is invariant under strictly increasingtransformations, say ψ and ζ, of the arguments of the cdf. That is, if

H(x, y)=C[F(x),G(y)]  (21)

then the transformed cdf has the same unique copula, C

H′(x, y)=H[ψ(x),ζ(y)]=C[F′(x),G′(y)],   (22)

where F′(x)=F(ψ(x)) and G′(y)=G′(ζ(y)).

A copula contains all the information of rank dependency of its marginalvariables. Kendall's tau is a statistic which summarizes this rankdependency. Tau is best understood by considering how to compute the“sample” tau from data (real or synthesized) per Eq. (3). Lessintuitively, the “population” tau can also be computed from theanalytical form of a copula using the known formula:

$\begin{matrix}\begin{matrix}{\tau = {{4{\int{\int_{I^{2}}{{C\left( {u,v} \right)}{{C\left( {u,v} \right)}}}}}} - 1}} \\{= {{4{\int_{0}^{1}{{u}{\int_{0}^{1}{{{{vC}\left( {u,v} \right)}}\frac{\partial^{2}{C\left( {u,v} \right)}}{{\partial u}{\partial v}}}}}}} - 1}}\end{matrix} & (23)\end{matrix}$

where the second equality shows the copula pdf (the mixed secondderivative) explicitly. It is important to note that Eq. (23) dependsonly on the copula, and not on the marginal distributions.

Although tau does not in general uniquely determine the copula, for afamily of copulas spanned by a single parameter, a sample estimate oftau from data can be used to determine the parameter. The parameter ofthe copula is determined by adjusting it to make the model populationvalue of tau from Eq. (23) match the sample estimate of tau. This givesan easy way to fit single-parameter copula models to data provided thatthe data is sampled from the entire population space. However, test datais typically highly censored. Remember that the conditions of the DRAMexperiment span only about 40 PPM near the origin of the copula. Thesample tau for the censored data can still be used to determine thecopula parameter if the model tau is computed only from part of thepopulation model copula corresponding to the censored data. If Test andUse right-censor x and y

x_(i)≦a, y_(i)≦b   (24)

then the “sub-population” model tau computed from a generalization ofEq. (23) for a restricted domain, [0, u_(a)≡F(a)]×[0, v_(b)≡G(b)] of [0,1]² is

$\begin{matrix}{\tau_{subpopulation} = {{\frac{4}{C^{2}\left( {u_{a},v_{b}} \right)}{\int_{0}^{u_{a}}{{u}{\int_{0}^{v_{b}}{{{{vC}\left( {u,v} \right)}}\frac{\partial^{2}{C\left( {u,v} \right)}}{{\partial u}{\partial v}}}}}}} - 1}} & (25)\end{matrix}$

which depends on the censor limits a and b, as well as the parameter(s)of the copula. For u_(a)=v_(b)=1 the censored sub-population is the sameas the entire population and Eq. (25) reduces to the population formulaof Eq. (23). Eq. (25) is a new result, derived below, which is part ofthe invention described here.

Analytical evaluation of Eq. (23) or Eq. (25) can be daunting. Accordingto one embodiment of the invention, an alternative method of computingtau is Monte-Carlo (MC) synthesis of data from a copula by any of manyknown methods, and evaluation of tau using Eq. (3). This is a good wayto evaluate the population tau corresponding to Eq. (23). But the MCevaluation of the copula model subpopulation tau corresponding to thehighly censored data can be very inefficient if the samples cannot beconfined to the region of the data, and many samples must be rejected.According to another embodiment of the invention, a geometrical copulaof the kind described in this invention provides a way to compute thesubpopulation tau by the MC method with complete efficiency becausesamples can be confined to any sub domain of the copula. This is nottrue of copulas in general.

The main challenge to use of copula methods is choosing a copula thatmakes sense for the particular application. In spite of the constraintson the functions which qualify as copulas, there are many sometimesexotic functions that are copulas. Many have interesting properties butthe properties are often not related to an obvious underlying stochasticmechanism. They may also have limited parametric flexibility to fitdata. For example, Archimedian copulas have nice algebraic properties,which Make them tractable and model-fitting methods have been developed,but it is often hard to relate them to a plausible underlying mechanism,and even to appreciate their geometrical shapes. A non-Archimedianexample is the Marshall-Olkin copula, which is a natural bi-variateextension of the Poisson process and so has an intuitive stochasticinterpretation. But the Marshall-Olkin copula is not flexible enough tofit many scenarios, including the DRAM example. A third example is theGaussian copula, which is a well-known extension of the multi-normaldistribution. As shown below, when the Gaussian copula is fitted to thedata of the DRAM example it had the drawback that its parameter wasforced to an implausible limit to fit the data.

Geometrical copulas offer an intuitive and practical approach to theproblem of choosing a copula for manufacturing test applications.Geometrical copulas define probability densities along easy to visualizelines and regions of the copula's domain. These shapes can be adjustedby parameters with geometrical interpretations. According to the currentinvention, test data is acquired over a range of environmentalconditions and sample sizes, which span the application over which themodel is used. This means that the copula needs to “look like” the data,and the fitted model will not be used to extrapolate far from the data.For the DRAM application example two geometrical copulas, the “stripe”and the “wedge”, in addition to the Gaussian copula are derived andfitted to the data.

An indicator that can also guide the choice of copula for testapplications is the limit

$\begin{matrix}{{LT} = {\lim\limits_{n->{0 +}}\frac{C\left( {u,u} \right)}{u}}} & (26)\end{matrix}$

which characterizes the lower tail dependence of C(u, v) near theorigin. It is seen by inspection of Eq. (14) that this limit vanishesfor copulas Π and W, but is unity for M Any finite value for this limitindicates that the copula has asymptotic dependence in the lower tail.Since one would expect dependence of retention times observed in Testand in Use to persist even for the few units, which have very shortretention times, a copula for which this limit is finite is a good modelcandidate for the DRAM example.

According to the invention, a major benefit of copula models is thatthey provide a deterministic semi-analytical way to compute figures ofmerit used to characterize, and so specify, the test manufacturingprocess. This is much more efficient than doing a Monte-Carlo (MC)synthesis from the models. But sometimes, MC synthesis is unavoidable,and the current invention provides MC synthesis in these cases. Anothermajor benefit of geometrical copulas (not shared with the Gaussiancopula, for example) is that it is possible to generate MC samples onlyin the important tail region, without wasting samples in the largerdomain of the copula. This will be shown for the geometrical copulasdescribed below.

Regarding Gaussian copulas, a method of modeling dependence is to treatvariables as multi-normally distributed correlated statisticalvariables. In the bi-normal case the cdf is

$\begin{matrix}{{H\left( {x,y} \right)} = {{\Phi_{2}\left( {x,{y;\rho}} \right)} = {\frac{1}{2\pi \sqrt{1 - \rho^{2}}}{\int_{- \infty}^{x}{\int_{- \infty}^{y}{{\exp \left\lbrack {- \frac{x^{\prime \; 2} - {2\rho \; x^{\prime}y^{\prime}} + y^{\prime \; 2}}{2\left( {1 - \rho^{2}} \right)}} \right\rbrack}\ {y^{\prime}}\ {x^{\prime}}}}}}}} & (27)\end{matrix}$

The correlation coefficient, −1≦ρ≦1, quantifies the dependence. TheGaussian copula corresponding to Eq. (27) is

Ga(u, v; ρ)=Φ(Φ₁ ⁻¹(u),Φ₁ ⁻¹(v); ρ)   (28)

where Φ₁ is the standard normal cdf.

Advantages of the Gaussian copula in the current invention include easyextension to higher dimensions, easy Monte Carlo simulation for theentire domain of the copula using Cholesky decomposition of thecovariance matrix, and availability of known numerical algorithms toevaluate Eq. (28). For test applications, however, efficient Monte Carlosample generation focused only on a part of the domain such as the tailregion near the origin is not possible. This is a significantdisadvantage in the test application compared to the geometrical copulasdescribed below. The fact that LT vanishes for the Gaussian copula,except for ρ=1, would tend to disqualify this copula for applicationsthat require non-vanishing correlation deep into the fail/fail tail ofthe multivariate distribution, such as the DRAM example. However,because the Gaussian copula is commonly used, it will be fitted to theDRAM data as an example which will make clear the benefits of the newgeometrical-copula-based method described in this invention. The relatedt-copula has a finite value of LT, and so may be more suitable than theGaussian copula, but it shares the other advantages and, particularlythe disadvantages, of the Gaussian copula for the test applicationsdescribed in this invention.

Regarding the stripe copula, the data in FIG. 6 is concentrated along adiagonal, with some scatter to either side. This suggests using a copulain which has finite probability density in a diagonal stripe on eitherside of the diagonal, but vanishing probability density outside thestripe. The width of the stripe can then be adjusted by a parameter.Both the shape of the stripe and the probability density are adjusted tomake the margins uniform so that it is a copula. The stripe copula isconstructed by first drawing a diagonal stripe across the unit squareshown in FIG. 13 to construct a stripe pseudo-copula. The parameter dcontrols the width of the stripe, which can range from zero (perfectcorrelation), to covering the entire unit square uniformly(independence). The probability density in the stripe is uniform,normalized to unity, and vanishes outside the stripe. The uniformprobability density of the stripe in FIG. 13 is 1/(2d−d²), thereciprocal of the stripe's area. A(u, v) is the function which gives theprobability enclosed by the rectangle (0, 0)/(u, v). By considering fourdistinct geometrical cases, expressions for the probability density A(u,v) enclosed by the boundaries of the rectangle (0, 0)/(u, v) shown inFIG. 13 may be found as a function of (u, v). All of these cases arecovered by the formula for the stripe pseudo-copula

$\begin{matrix}{{{A\left( {u,v} \right)} = \frac{{u^{\prime}v^{\prime}} - {\frac{1}{2}a^{2}} - {\frac{1}{2}b^{2}}}{d\left( {2 - d} \right)}}{where}} & (29) \\{{u^{\prime} = {\min \left\lbrack {u,{v + d}} \right\rbrack}}{v^{\prime} = {\min \left\lbrack {v,{u + d}} \right\rbrack}}{a = {\max \left\lbrack {{u^{\prime} - d},0} \right\rbrack}}{b = {{\max \left\lbrack {{v^{\prime} - d},0} \right\rbrack}.}}} & (30)\end{matrix}$

The function A is a pseudo-copula because it satisfies the requirementsof a copula except that the margins are not uniform. The marginaldistributions, A(u, 1)=f⁻¹(u) and A(1, v)=f⁻¹(v), are non-uniform sincef⁻¹(z)≈z. z is a dummy argument which can be either u, or v. Thefunction f ⁻¹(z) is:

Case d≦½

$\begin{matrix}{{f^{- 1}(z)} = {\frac{1}{d\left( {2 - d} \right)} \times \left\{ {{\begin{matrix}{{\frac{1}{2}z^{2}} + {zd}} & {0 \leq z \leq d} \\{{2{zd}} - {\frac{1}{2}d^{2}}} & {d \leq z \leq {1 - d}} \\{z - {\frac{1}{2}\left( {z - d} \right)^{2}} - {\frac{1}{2}\left( {1 - d} \right)^{2}}} & {{1 - d} \leq z \leq 1}\end{matrix}\mspace{20mu} {Case}\mspace{14mu} d} \geq \frac{1}{2}} \right.}} & (31) \\{{f^{- 1}(z)} = {\frac{1}{d\left( {2 - d} \right)} \times \left\{ \begin{matrix}{{\frac{1}{2}z^{2}} + {zd}} & {0 \leq z \leq {1 - d}} \\{z - {\frac{1}{2}\left( {1 - d} \right)^{2}}} & {{1 - d} \leq z \leq d} \\{z - {\frac{1}{2}\left( {z - d} \right)^{2}} - {\frac{1}{2}\left( {1 - d} \right)^{2}}} & {d \leq z \leq 1}\end{matrix} \right.}} & (32)\end{matrix}$

To construct the stripe copula from the stripe pseudo-copula the inverseof this function is needed.

$\begin{matrix}{\mspace{79mu} {{{{Case}\mspace{14mu} d} \leq \frac{1}{2}}{{f(z)} = \left\{ \begin{matrix}{{- d} + \sqrt{d^{2} + {2\left( {{2d} - d^{2}} \right)z}}} & {0 \leq z \leq \frac{3d}{4 - {2d}}} \\{{\left( {1 - {\frac{1}{2}d}} \right)z} + {\frac{1}{4}d}} & {\frac{3d}{4 - {2d}} \leq z \leq \frac{4 - {5d}}{4 - {2d}}} \\{1 + d - \sqrt{{2\left( {{2d} - d^{2}} \right)\left( {1 - z} \right)} + d^{2}}} & {\frac{4 - {5d}}{4 - {2d}} \leq z \leq 1}\end{matrix} \right.}}} & (33) \\{\mspace{79mu} {{{{Case}\mspace{14mu} d} \geq \frac{1}{2}}{{f(z)} = \left\{ \begin{matrix}{{- d} + \sqrt{d^{2} + {2\left( {{2d} - d^{2}} \right)z}}} & {0 \leq z \leq \frac{1 - d^{2}}{2\left( {{2d} - d^{2}} \right)}} \\{{\left( {{2d} - d^{2}} \right)z} + {\frac{1}{2}\left( {1 - d} \right)^{2}}} & {\frac{1 - d^{2}}{2\left( {{2d} - d^{2}} \right)} \leq z \leq {1 - \frac{1 - d^{2}}{2\left( {{2d} - d^{2}} \right)}}} \\{1 + d - \sqrt{{2\left( {{2d} - d^{2}} \right)\left( {1 - z} \right)} + d^{2}}} & {{1 - \frac{1 - d^{2}}{2\left( {{2d} - d^{2}} \right)}} \leq z \leq 1}\end{matrix} \right.}}} & (34)\end{matrix}$

So the stripe copula sought is

St(x, y)=A(f(x),f(y)).   (35)

This is a copula because it has all the properties of a copula,including uniform margins:

St(x, 1)=A(f(x),f(1))=A(f(x),1)=f ⁻¹(f(x))=x   (36)

since x is the cdf of the uniform distribution. The equation of theupper line bounding the area of finite probability density for thiscopula is

$\begin{matrix}{y = \left\{ \begin{matrix}{f^{- 1}\left( {{f(x)} + d} \right)} & {0 \leq x \leq {f^{- 1}\left( {1 - d} \right)}} \\1 & {{f^{- 1}\left( {1 - d} \right)} \leq x \leq 1}\end{matrix} \right.} & \;\end{matrix}$

and the equation of the lower line is

$\begin{matrix}{y = \left\{ \begin{matrix}0 & {0 \leq x \leq {f^{- 1}(d)}} \\{f^{- 1}\left( {{f(x)} - d} \right)} & {{f^{- 1}(d)} \leq x \leq 1.}\end{matrix} \right.} & (38)\end{matrix}$

The low tail dependence is

$\begin{matrix}{{LT} = {{\lim\limits_{x\rightarrow{0 +}}\frac{{St}\left( {x,x} \right)}{x}} = {{\lim\limits_{x\rightarrow{0 +}}\frac{f^{2}(x)}{{xd}\left( {2 - d} \right)}} = {{\lim\limits_{x\rightarrow{0 +}}{\frac{2 - d}{d}x}} = {0\mspace{31mu} \left( {d > 0} \right)}}}}} & (39)\end{matrix}$

so there is no asymptotic low tail dependence unless d vanishes.

Algorithms for Monte-Carlo synthesis of random points in geometricalcopulas such as the stripe copula can be derived using geometricalarguments starting with the pseudo copula, A. It is possible to fill anyparallelogram or triangle with uniformly distributed random points usingevery point generated, that is, without the rejection of any Monte-Carlosample. This is done by weighting the basis vectors that define theparallelogram with a pair of independent uniformly distributed randomnumbers. For a triangle, points in the “wrong half” of a parallelogramare reflected into the triangle of interest. The slice in FIG. 13 can bedecomposed into rectangles and triangles, and random points placed inthem according to probabilities determined by area ratios of therectangles and triangles.

This produces uniformly distributed points within the stripepseudo-copula. Most importantly, it is possible to limit the region ofthe stripe over which these points are generated. Generated points maybe mapped into the stripe copula using Eqs. (31) and (32), so that if u,and v are generated for the pseudo-copula, then the corresponding pointsof the copula are

x=f ⁻¹(u), y=f ⁻¹(v).   (40)

FIGS. 14 a-14 f show examples of synthesized probability maps of thestripe copula. The density of random points indicates the probabilitydensity of the copula. Notice that both the shape and the density ofpoints have been transformed by the function f, Eqs. (33) and (34), fromthe pseudo-copula of FIG. 13. FIGS. 14 a-14 c show that the stripecopula spans independence (d=1) to perfect correlation (d=0), and FIGS.14 d-14 f show synthesized points concentrated near the origin. Thedegree of censoring, or censor fraction, is indicated in FIGS. 14 a-f bythe parameter “Censor”. The censor fraction is the number of units in asample or sub-population divided by the number of units in the entirepopulation. Kendall's tau is computed from the synthesized data usingEq. (3) and is also shown in the header of plots in FIGS. 14 a-f. Recallfrom Table 3 that DRAM bits for which retention times were measuredcovered only 40 PPM of the population. This degree of censoring, typicalof integrated circuit test applications, would be a tiny dot near theorigin on the scale of the plots in FIGS. 14 a-f. The efficiencyadvantage of any Monte Carlo (MC) sample generation method whichconfines samples to the non-censored subpopulation, compared to a MCsample generation method which must reject samples of the entirepopulation, is proportional to the reciprocal of the censor fraction(“Censor” in FIGS. 14 a-f). So the small censor fractions typical oftest applications gives a very large MC efficiency advantage to thegeometrical copula method of selective sample generation described inthis invention.

Regarding the wedge copula, the data in FIG. 6 appears to be morescattered as retention time increases. This suggests constructing awedge-shaped copula. The construction starts with a wedge-shapedpseudo-copula A(u, v) symmetrical about the (0, 0)/(1, 1) diagonal withuniform probability density inside the wedge, and vanishing probabilitydensity outside the wedge, shown in FIG. 15. This shaded region has area(1−c)/c, where c is defined in the figure, so the wedge's uniformprobability density is c/(1−c). The size of the region is controlled bythe parameter 1≦c≦∞ which ranges from c=1 for perfect correlation and toc=∞ for independence.

The wedge pseudo-copula can be shown by geometrical considerations to bethe probability density enclosed by the (0, 0)/(u, v) rectangle in FIG.15 for (u, v) anywhere in [0, 1]²

$\begin{matrix}{{{A\left( {u,v} \right)} = {\frac{c}{c - 1}\left( {{u^{\prime}v^{\prime}} - \frac{u^{\prime \; 2}}{2c} - \frac{v^{\prime \; 2}}{2c}} \right)}}{{u^{\prime} = {\min \left\lbrack {u,{cv}} \right\rbrack}},{v^{\prime} = {\min \left\lbrack {v,{cu}} \right\rbrack}}}} & (41)\end{matrix}$

On the margins, this is

$\begin{matrix}{{{A\left( {u,1} \right)} = {f^{- 1}(u)}}{{A\left( {1,v} \right)} = {f^{- 1}(v)}}{where}} & (42) \\{{f^{- 1}(z)} = \left\{ \begin{matrix}\frac{\left( {c + 1} \right)z^{2}}{2} & {0 \leq z \leq c^{- 1}} \\{z - \frac{\left( {1 - z} \right)^{2}}{2\left( {c - 1} \right)}} & {c^{- 1} \leq z \leq 1}\end{matrix} \right.} & (43)\end{matrix}$

which is a monotonically increasing function of the dummy argument, z.Notice that A(u, v) is a pseudo-copula because the marginaldistributions, Eqs. (42), are not uniform. To construct the wedge copulathe inverse of Eq. (43) is needed:

$\begin{matrix}{{f(z)} = \left\{ \begin{matrix}\sqrt{\frac{2z}{c + 1}} & {0 \leq z \leq \frac{1 + c}{2c^{2}}} \\{c - \sqrt{\left( {c - 1} \right)^{2} + {2\left( {c - 1} \right)\left( {1 - z} \right)}}} & {\frac{1 + c}{2c^{2}} \leq z \leq 1}\end{matrix} \right.} & (44)\end{matrix}$

So the wedge copula corresponding to FIG. 15 is

We(x, y)=A(f(x),f(y))   (45)

which satisfies the requirement that it have uniform marginaldistributions because

We(x, 1)=A(f(x),f(1))=A(f(x),1)=f ⁻¹(f(x))=x   (46)

and the same for y, since x and y are cdfs of uniform distributions.

The lower boundary of the area of non-vanishing probability is

y=f ⁻¹(c ⁻¹ f(x)),0≦x≦1   (47)

while the upper boundary is

$\begin{matrix}{y = \left\{ \begin{matrix}{f^{- 1}\left( {{cf}(x)} \right)} & {0 \leq x \leq {\frac{1}{2}{\left( {c + 1} \right)/c^{2}}}} \\1 & {{\frac{1}{2}{\left( {c + 1} \right)/c^{2}}} \leq x \leq 1.}\end{matrix} \right.} & (48)\end{matrix}$

For the test application the important region is the region near theorigin. In this region the boundaries of the regions with non-vanishingprobability given by Eqs. (47) and (48) are straight lines. The lowerboundary is

$\begin{matrix}{{y = {c^{- 2}x}}{0 \leq x \leq \frac{c + 1}{2c^{2}}}} & (49)\end{matrix}$

and the upper boundary is

$\begin{matrix}{{y = {c^{2}x}}{0 \leq x \leq {\frac{c + 1}{2c^{4}}.}}} & (50)\end{matrix}$

The low tail dependence of the wedge copula is

$\begin{matrix}{{LT} = {{\underset{x\rightarrow{0 +}}{\lim \;}\frac{{We}\left( {x,x} \right)}{x}} = {{\lim\limits_{x\rightarrow{0 +}}\frac{f^{2}(x)}{x}} = {{\lim\limits_{x\rightarrow{0 +}}{\frac{1}{x}\frac{2x}{c + 1}}} = \frac{2}{c + 1}}}}} & (51)\end{matrix}$

So the wedge copula has asymptotic tail dependence except in the limitof independence (c→∞) where it becomes Π, the independence copula. Inthe opposite limit (c→1) the asymptotic dependence becomes unity becauseWe becomes M, the Frechet upper bound copula corresponding to perfectcorrelation.

Points of the wedge copula may be synthesized by generating uniformlydistributed (u, v) points in the wedge-shaped finite uniform probabilitydensity area of the pseudo-copula A in FIG. 15, and then mapping them tothe space of the copula using x=f(u), and y=f(v) where f is given by Eq.(44). The algorithm for generating (u, v) points is given below. Theprobability density maps of the wedge copula in FIGS. 16 a-16 f weregenerated using this method. Notice that it is possible to restrict thegeneration of synthesized points to sub-domains of the copula,particularly the region near the origin. FIGS. 16 a-16 c show that thewedge copula spans independence (c=∞) to perfect correlation (c=1), andFIGS. 16 d-16 f show that synthesized points can be concentrated nearthe origin, corresponding to values of the censor fraction, “Censor” <1.

Values of Kendall's tau given in FIGS. 16 a-16 f for the synthesizeddata of the wedge copula were computed to good precision using Eq. (3).It is also possible to derive an analytical expression for Kendall's taufor the wedge copula as a function of the parameter c, and of the censorfraction a, using Eq. (25):

$\begin{matrix}{{\tau_{Subpopulation}\left( {c,a} \right)} = {{\frac{4}{{We}^{2}\left( {a,{a;c}} \right)}{\int_{0}^{a}{{x}{\int_{0}^{a}{{{{yWe}\left( {x,{y;c}} \right)}}\frac{\partial^{2}{{We}\left( {x,{y;c}} \right)}}{{\partial x}{\partial y}}}}}}} - 1}} & (52)\end{matrix}$

The result, derived below, is

$\begin{matrix}{{\tau_{Subpopulation}\left( {c,a} \right)} = \frac{{2c} + 1}{3c^{2}}} & (53)\end{matrix}$

The wedge copula has the attractive property that the subpopulation tauis independent of a, the censor fraction. This is not true of copulas ingeneral as may be seen by comparing the cases in FIGS. 14 a-14 f andFIGS. 16 a-16 f for which the parameter d or c, respectively, isconstant, but the censor fraction varies. Since tau may be computeddirectly from data, Eq. (53) provides a way to estimate the parameter cof the wedge copula, according to the current invention.

Since it is convenient if the model subpopulation tau is independent ofthe censor fraction, it is useful to know more general conditions underwhich this holds so that copulas with this property can be identified. Asufficient condition for this to be true is that a copula be expressedas C(x, y)=A[f(x),f(y)] where A satisfies A(a×u, a×v)=a²×A(u, v), for a≦1. This is shown below. The geometrical interpretation is that allsub-regions [0, a]² of the pseudo-copula A are geometricallyself-similar. It is immediately apparent by inspection of FIG. 13 thatthis is not true of the stripe copula, whereas FIG. 15 shows that it istrue of the wedge copula.

Turning now to an example of fitting DRAM data to copula models, theselection of a copula is guided by the conditions of data acquisitionand specific knowledge of integrated circuit manufacturing. The DRAMretention time data is deliberately symmetrical between Test and Usebecause the selection of a symmetrical test coverage model for thepurpose of model-fitting maps r_(min) and r_(max) for each bit ton_(test) and r_(use) with equal probability. So only “exchangeable”[C(x, y)=C(y, x)] model copulas need to be considered. All of thecandidate model copulas are exchangeable. According to one embodiment ofthe invention, if the dependent attributes of a test vehicle or productare expected to be related only by the intrinsic physics of thematerials and design, and defects will affect one or the other but notboth of the attributes, then one can expect that dependency will bestrong in the bulk but weak in the tails of the two-dimensionaldistribution. In this example case, a copula such as the Gaussiancopula, for which LT=0, will be suitable. If, on the other hand, adefect is regarded as potentially affecting both attributes, as in theDRAM case, then the dependency will extend deep into “fail/fail” tail ofthe two-dimensional distribution. In this case, one would choose acopula model which can have a finite lower tail dependence, LT≈0. Sinceboth the Gaussian copula and the stripe copula have LT=0 except inlimiting cases, and the data suggest that a model with LT≈0 will beneeded, a two-parameter copula model was constructed using a linearcombination of each of the wedge, Gaussian, and stripe copulas with theFrechet upper bound copula, M The model is

C′(x, y, p, parameter)=pM(x, y)+(1−p)C(x, y; parameter)   (54)

where the parameter is ρ, d, or c when C is the Ga, St, or We copuladescribed above. This model C′ will have LT≈0 if p≈0. It may have LT≈0for vanishing p, if C has LT≈0 by itself.

A least-squares method was used to extract the best fit copula at eachof the 18 environmental conditions for each skew of the DRAM data. Thefollowing sum was minimized using Excel's solver to determine p, and thecopula-specific model parameter:

$\begin{matrix}{{{{SSQ}\left( {p,{parameter}} \right)} = \frac{\sum\limits_{i}^{\;}{\sum\limits_{j}^{\;}\left( {{N \times \delta \; {C_{ij}^{\prime}\left( {p,{parameter}} \right)}} - n_{ij}} \right)^{2}}}{\sum\limits_{i}^{\;}{\sum\limits_{j}^{\;}n_{ij}^{2}}}}{where}} & (53) \\{{\delta \; C_{ij}^{\prime}} = {C_{ij}^{\prime} - C_{{i - 1},j}^{\prime} - C_{i,{j - 1}}^{\prime} + C_{{i - 1},{j - 1}}^{\prime}}} & (56)\end{matrix}$

where i, and j are cell indexes in FIG. 6, where n_(ij) is the number offailures observed in each cell, and where N=48750000 is the totalpopulation size.

In every case the best fit was found at p=0. So, for the DRAM, aone-parameter model suffices for any of the three candidate copulas.

For a one-parameter copula, a simpler and more convenient way than theleast-squares method to extract the parameter of the copula is tocompute a “sample” estimate of Kendall's tau directly from the data andcompare it with the theoretical expression for the subpopulation tau,Eq. (25), to solve for the parameter. The theoretical expression for thesubpopulation tau is generally also a function of the censor fraction,which is known from the experiment.

Regarding model parameter estimation for the wedge copula, an estimateof c may be derived by substituting the sample tau computed from thedata using Eq. (3), and given in Table 4, into the inverse of Eq. (53),written as

$\begin{matrix}{c = {\frac{1 + \sqrt{1 + {3\tau}}}{3\tau}.}} & (57)\end{matrix}$

Generally a relation like Eq. (57) depends on the censor fraction butfor the wedge copula there is no censor fraction dependence. FIG. 17 ashows estimates of c as a function of the 18 environmental conditions,lnα. Estimates using Eq. (57) and the more generally applicableleast-squares method show good agreement in the figure. Moreover, thefigure shows that c is independent of the environmental condition.

Regarding the Gaussian copula, FIG. 17 b shows a reasonablyenvironmental-condition-independent fit to the Gaussian copula by theleast-squares method. It was not possible to determine the copulaparameter using the sample tau method because there is no easily derivedexpression from Eq. (25), nor is it possible to do an efficientMonte-Carlo computation of the sample tau because there is no known wayto confine random samples to the tail of the copula without rejection ofpoints. Remember that the range of the data covers only 40 PPM of thepopulation (for the nominal skew), so sampling the entire population toestimate tau for the range of the data is not practical. The bestleast-squares fit was obtained for a ‘correlation coefficient of about0.999 (see FIG. 17 b), with the significant variation occurring in the3^(rd) and higher digits. It seems that the main, unobserved, body ofthe population must be forced to have an extremely strong correlation,so that the weaker correlation in the tail, where the data is observed,will be sufficiently strong to fit the data. So, the Gaussian copula isnot a natural fit to the strong dependence observed in the range of thedata:

Regarding the stripe copula, FIG. 17 c shows parameter estimates for thestripe copula as a function of environmental condition, lnα, for boththe least-squares and the sample tau methods. Because a single parametervalue could not be chosen for all environmental conditions, no singlevalue of the stripe copula parameter can be given in Table 4. The strongenvironmental dependence of the copula parameter makes this anundesirable copula for the DRAM application. The sample-tau methodrequires an efficient way to compute Eq. (25) for the model copula. Forthe stripe copula, derivation of an analytical formula was daunting.But, as for any geometrical copula, it was possible to do very efficientand accurate Monte-Carlo numerical calculations of the sample taubecause all the samples may be concentrated in the sample region ofinterest. This is the easiest method for most geometrical copulas.

The quality of fit for each of the three types of copula is compared inFIG. 18, which shows a comparison of minimum sums of squares computedfrom Eq. (55) for each of 18 environmental conditions for the nominalskew. Most of the plotted points in FIG. 18 are above the diagonal,showing that the wedge copula gives the best fit.

The discussion shows application of several principles of copulaselection, which led to the selection of the wedge copula to model theDRAM application example:

-   -   Copula models with parameters independent of environmental        conditions are preferred because they greatly simplify the        model, and probably reflect a fundamental underlying dependency        structure of the mechanism. Moreover, an        environmentally-independent copula model in combination with        marginal distributions that embody all Test and Use conditions        in a single parameter such as r/α, enables an “equivalent set        point” test method which gives flexibility in determining test        and datasheet specifications. For the DRAM example, the wedge        copula was environmentally independent, the Gaussian copulas was        less so, and the stripe copula was strongly environmentally        dependent.    -   Copula models with a plausible tail dependence based on        application knowledge are desirable. Geometrical copulas are        preferred because they are easy to construct to mimic observed        or plausible behavior. Only the wedge copula satisfies this for        the DRAM example. Comparison of FIG. 6 with FIGS. 19 a-19 b show        that the best-fit wedge copula is a better representation of the        shape of the tail dependence of the data than the best-fit        stripe copula. FIGS. 19 a-19 b were synthesized from the best        fit wedge and slice copulas at the highest environmental        condition of the nominal skew. The sample size of 1641 and        censoring also matched the experimental conditions of FIG. 6.    -   Geometrical copulas are also preferred because it is possible to        concentrate Monte-Carlo samples in the tail region of a        geometrical copula. This has two major benefits. First, highly        accurate Monte-Carlo evaluations of the sample tau can be        obtained, sidestepping the need for deriving analytical formulas        from Eq. (25). Second, any Monte-Carlo simulation from the        copula can be extremely efficient since sample generation can be        focused on the tail region. Both wedge and stripe copulas have        this advantage.

Based on these considerations, the wedge copula model was selected tomodel the dependency structure of the DRAM bit retention time.

Turning now to the aspect of the invention in which the copula-basedstatistical model fitted to the data according to the precedingdescription is used to model Test and Use of a integrated circuitproduct, FIG. 20 shows how good and bad (defective) units of anintegrated circuit product such as a memory array are produced by theFab/Assembly process, and then are screened by final Test and go on toUse. The figure also shows important figures of merit (FOMs) includingYield Loss, Overkill (there are two kinds), and End-Use Fail Fraction.

Regarding the FOMs, targets and cost models, a schematic drawing of howthe Test and Use conditions divide the population of manufactured unitsinto categories is shown in FIG. 21. The three proportions shown in FIG.21, which are modeled as probabilities, are sufficient to characterizethe effect of Test and Use. The four FOMs described next are defined interms of these probabilities.

First, Yield Loss is given by

YL=P(Fails Test)=1P(Passes Test)   (58)

where P(Passes Test) is the fraction of units which pass Testirrespective of whether they are good or bad in Use. Yield Loss is aprimary manufacturing indicator since it directly affects revenue.

Second, End Use Fail Fraction is given by

$\begin{matrix}\begin{matrix}{{EUFF} = {P\left( {{Fails}\mspace{14mu} {in}\mspace{14mu} {Use}} \middle| {{Passes}\mspace{14mu} {Test}} \right)}} \\{= {1 - {P\left( {{Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \middle| {{Passes}\mspace{14mu} {Test}} \right)}}} \\{= {1 - \frac{P\left( {{Passes}\mspace{14mu} {Test}\mspace{14mu} {and}\mspace{14mu} {Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)}{P\left( {{Passes}\mspace{14mu} {Test}} \right)}}} \\{= {\frac{{P\left( {{Passes}\mspace{14mu} {Test}} \right)} - {P\left( {{Passes}\mspace{14mu} {Test}\mspace{14mu} {and}\mspace{14mu} {Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)}}{P\left( {{Passes}\mspace{14mu} {Test}} \right)}.}}\end{matrix} & (59)\end{matrix}$

EUFF is the fraction of units classified as failing in Use, given thatthey have passed Test (a conditional probability). End-Use Fail Fraction(EUFF) is a primary quality indicator since it is the customer-perceivedproportion of defective units.

Third, Manufacturing Overkill is given by

OKill(Mfg)=P(Good in Use)−P(Passes Test and Good in Use).   (60)

Finally, Test Overkill is given by

$\begin{matrix}{{{OKill}({Test})} = {\frac{{P\left( {{Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)} - {P\left( {{Passes}\mspace{14mu} {Test}\mspace{14mu} {and}\mspace{14mu} {Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)}}{1 - {P\left( {{Passes}\mspace{14mu} {Test}} \right)}}.}} & (61)\end{matrix}$

Overkill is associated with the cost of missed opportunity for revenuecaused by invalidly rejecting units at Test. FIGS. 20-21 shows thatManufacturing Overkill, Eq. (60), is the fraction of all manufacturedunits (good and bad in Use), which are invalidly rejected by Test andthat Manufacturing Overkill is a subset of Yield Loss. ManufacturingOverkill is a measure of the cost to the entire manufacturing process.Test Overkill, Eq. (61), is the fraction of all units rejected by testwhich are invalidly rejected by Test and is a measure of how good theTest screen is.

Turning to consider targets, the definitions of FOMs show that each FOMis a ratio falling into the range [0, 1] and that for each, “0” is mostdesirable and “1” is least desirable. So specifications for a productand test manufacturing process are found by requiring that FOMs meetdo-not-exceed targets for each FOM. Targets are chosen with producercosts and customer-perception of brand image in mind, and areproduct-specific. Quality-related customer costs are one aspect of thisperception. Example targets used in the application examples later inthe description of the invention are YL(Target)=20%, OKill(Mfg,Target)=2%, EUFF (Target)=200 DPPM. The values of the example targetsare not representative of any particular product or manufacturingprocess.

Regarding the Datasheet Specification and Test Condition shown in FIG.20, one embodiment of the invention (the DRAM example) specifies thesein terms of environmental conditions (V_(p), V_(d), T) and retentiontime, r. The settings of these four parameters in Use (DatasheetSpecification) and at Test (Test Condition) will be different. Usuallythe Test Condition is more “stressful” than the Datasheet Specification.Because of the fit of environmental conditions (V_(p), V_(d), T) to themodel of Eq. (2), the environmental condition is mapped into the singleparameter α_(use) in Use and α_(test) in Test. The datasheet specifies arefresh time r_(use), and the Test specification uses a differentretention time limit setting, r_(test). So, a single parameter, x,depending on both the environmental condition via α and retention timelimit r_(use) in the datasheet defines the Datasheet Specification (Use)condition:

$\begin{matrix}{x = {1 - {\exp \left\lbrack {- \left( \frac{r_{use}}{\alpha_{use}} \right)^{\beta}} \right\rbrack}}} & (62)\end{matrix}$

And similarly, a single parameter, y, defines the Test Condition:

$\begin{matrix}{y = {1 - {{\exp \left\lbrack {- \left( \frac{r_{test}}{\alpha_{test}} \right)^{\beta}} \right\rbrack}.}}} & (63)\end{matrix}$

Figures of merit are used in cost models of the integrated circuitproduct. Costs to the producer of a component and costs to the customerproducing systems using the component must be included in these models.Regarding producer costs, suppose the cost of manufacturing a componentis $Cost and the sale price of a component is $Price, and suppose Ncomponent units are to be manufactured. Besides costs of materials usedin the unit, $Cost includes per-unit capital depreciation costsassociated with the manufacturing equipment such as testers, charged tothe unit. For testers, this cost-contributor will depend on the timeneeded to test each unit, among other factors. According to thedefinitions of the FOMs defined above, the cost of manufacturing Ncomponent units is

Cost of manufacturing N units=N×$Cost   (64)

and the revenue from the N units manufactured is

Revenue from N units manufactured=N×(1−YL)×$Price−N×OKill(Mfg)×$Price  (65)

where OKill (Mfg) is the fraction of manufactured units which would havebeen good in Use, but were rejected at Test.

If the revenue reduced by overkill is regarded as an opportunity costthen the per-unit-manufactured cost is

$Cost′=$Cost+OKill(Mfg)×$Price.   (66)

and the revenue is

Revenue from units manufactured=N×(1−YL)×$Price   (67)

For products with high margins so that $Price>>$Cost, and withsignificant yield loss, overkill may significantly affect the businessviability of a product.

Regarding customer costs, the producer's price per component unit is thecustomer's nominal cost per unit plus additional costs to the customerdue to debug and rework or scrapping of systems which fail because offaulty components which have escaped the component manufacturer'sprocess. It is also possible that some faulty components may escape thecustomer's testing and lead to warranty costs. So, for the customer

Cost of components=N×$Price+N×EUFF×$Average cost impact of defectiveunit to customer.   (68)

The cost impact of a defective component to a system manufacturer isusually much greater than the price of the component, particularly forsurface-mounted components. Therefore, EUFF is typically required to beless than about 200 DPPM. Beyond cost, EUFF has a qualitative impact onbrand image. This is often the primary consideration in choosing theEUFF target.

Turning now to consider the test coverage model, note that for thecopula-based statistical model fitted to the data, the test coveragemodel used assumed that the minimum and maximum retention times for eachbit are equally likely to occur in Test and in Use. This is called a“symmetrical test coverage model”. However, for a realisticManufacture/Test/Use flow like FIG. 20, Use occurs over extended timeperiods, so that if a minimum retention time can occur for a bit it willcertainly occur in Use. On the other hand, Test is brief so theprobability of occurrence of the minimum or the maximum retention timeof a bit in Test depends on details of time-in-state of bit-leakagewhich are beyond what can be known from the DRAM example data. So anassumption must be made in order to proceed to model theManufacture/Test/Use flow of FIG. 20. The most conservative assumptionfrom the end-user perspective, called the “conservative test coveragemodel”, is that Use always “sees” the minimum retention time and thatTest always “sees” the maximum retention time. Other test coverage modelassumptions are also used to model Test and Use in order to gauge thesensitivity of the model to this assumption.

The test vehicle data were fitted assuming a symmetrical test coveragemodel, in which Test and Use are equivalent, by assigning r_(min) andr_(max) to Test or Use using a computer-generated “coin flip”. Soidentical marginal distributions for Test and Use were extracted fromthe data, and an exchangeable copula C(x, y) was fitted. If Test and Useretention times sampled from this symmetrical model, indicated by theprimes, are assigned to Test and Use under the conservative testcoverage model assumption that Test always “sees” r_(max) and Use always“sees” r_(min) then

r _(test)=max[r′ _(test) , r′ _(use) ], r _(use)=min[r′_(test),r′_(use)]  (69)

That is, r_(test) is the 2:2 order statistic and r_(use) is the 1:2order statistic of the pair (r′_(test), r′_(use)). It is known that the2-dimensional cdf connecting the 2:2, and 1:2 order statistics of a pairof random variables distributed according to a 2-dimensional cdf H(u, v)is

$\begin{matrix}{{K\left( {u,v} \right)} = \left\{ \begin{matrix}{{H\left( {u,v} \right)} + {H\left( {v,u} \right)} - {H\left( {u,u} \right)}} & {u \leq v} \\{H\left( {v,v} \right)} & {u \geq v}\end{matrix} \right.} & (70)\end{matrix}$

and that if the marginal distributions of H are the same, then thepseudo-copula of the order statistics has the same form as K. If inaddition, the copula of the original random variables is exchangeable sothat C(x, y)=C(y, x), then

$\begin{matrix}{{D\left( {x,y} \right)} = \left\{ \begin{matrix}{{2{C\left( {x,y} \right)}} - {C\left( {x,x} \right)}} & {x \leq y} \\{C\left( {y,y} \right)} & {x \geq {y.}}\end{matrix} \right.} & (71)\end{matrix}$

So D is the transformation of the fitted copula, C, embodying theconservative test coverage model that Test always “sees” r_(max) and Usealways “sees” r_(min).

Notice that the x and y marginal distributions are cdfs of the 1:2 and2:2 order statistics of pairs of numbers sampled from X and Y:

D(x,1)=2x−C(x, x)

D(1, y)=C(y, y)   (72)

This shows that in general D is a pseudo-copula, not a copula, becauseD(x, 1)≈x and D(1, y)≈y.

An integrated circuit DRAM memory includes an array of many, N, bits.The preceding description gives a model of the dependence structure, D,of retention times for a single bit. Needed is a model of the dependencestructure of retention times for an N-bit array.

Derived next is the dependence structure of N-bit arrays which are goodonly when all N bits are good. The more realistic and complex cases whenan array can be good if some bits are bad (fault tolerance) and whenarrays with bad bits can be repaired at Test will be described later.The probability that every bit in an array of N bits is good when theTest and Use conditions are x and y is the survival pseudo-copula of asingle bit, S, raised to the power of N

S _(N)( x, y )=[S( x, y)]^(N)=[1−D(1− x, 1)−D(1,1− y )+D(1− x,1− y)]^(N)   (73)

where x=1−x and y=1−y. S_(N) is the survival pseudo-copula of the array,so a little manipulation using D(x, 1)+D(1−x, 1)=1 and similarly for ygives the pseudo-copula of the non-fault-tolerant array as

$\begin{matrix}\begin{matrix}{{D_{N}\left( {x,y} \right)} = {1 - {S_{N}\left( {{1 - x},1} \right)} - {S_{N}\left( {1,{1 - y}} \right)} + {S_{N}\left( {{1 - x},{1 - y}} \right)}}} \\{= {1 - \left\lbrack {1 - {D\left( {x,1} \right)}} \right\rbrack^{N} - \left\lbrack {1 - {D\left( {1,y} \right)}} \right\rbrack^{N} +}} \\{{\left\lbrack {1 - {D\left( {x,1} \right)} - {D\left( {1,y} \right)} + {D\left( {x,y} \right)}} \right\rbrack^{N}.}}\end{matrix} & (74)\end{matrix}$

Notice that D_(N)=D when N=1, as it must.

Eq. (73) and (74) can be generalized in an obvious way to get thesurvival copula of a product with multiple copies of several types ofmodule for which copula models were extracted from test vehicle data.(For the DRAM, there is one type of module; the bit.) In particular, Eq.(73) becomes a product over module types with the survival copula ofeach module raised to a power which is the number of modules of thattype in the product.

The FOMs of interest for the non-fault-tolerant array can now bedescribed. FIG. 22 shows a schematic representation of how the Test andUse conditions in Eqs. (62) and (63) superimposed over the bitpseudo-copula pdf of Eq. (71) divide the population probability spaceinto four regions labeled according to a bit's Use/Test pass/failcategory. This depiction is schematic because, for the DRAM, x and y aremuch closer to the origin than shown. It will be shown that FOMs can beexpressed in terms of the probability mass (shaded) enclosed in each ofthe four labeled regions of the population probability space, and theprobability masses can be expressed in terms of the pseudo-copula D. Theessential tradeoff between overkill and end use fraction fail can beseen in FIG. 22 since moving the intersection of the Test and Usecondition settings to reduce overkill (reduce probability density inregion pf) will necessarily increase the probability density in regionfp, and so increase end use fail fraction. For any given Test conditionand Use condition each bit in a given array will fall into one of thefour categories shown in FIG. 22. For an N-bit array the sameconsiderations apply but the array pseudo-copula D_(N) given by Eq. (74)is used rather than the bit pseudo-copula D given by Eq. (71). So, foran N-bit array which is good only when all of its bits are good (thatis, it has no fault tolerance) the probability of occurrence of eachcategory of the array is

P _(fp) =D _(N)(x, 1)−D_(N)(x, y)

P _(pf) =D _(N)(1, y)−D _(N)(x, y)

P _(ff) =D _(N)(x, y)

P _(pp)=1−D_(N)(x, 1)−D_(N)(1, y)+D_(N)(x, y)   (75)

where the rules for writing the probability mass of a region of apseudo-copula, Eq. (18), have been used. Note that p_(fp)+p_(pf)+p_(ff)+p_(pp)=1. Keep in mind that, as shown in FIG. 22, the firstsubscript for a probability mass like p_(fp) refers to Use, and thesecond refers to Test.

Probabilities used to define FOMs may be written

P(Passes Test)=p _(pp) +p _(fp)

P(Good in Use)=p _(pp) +p _(pf)

P(Passes Test and Good in Use)=p _(pp)   (76)

so using Eqs. (58), (59), (60), (61), and (75) expressions for FOMs interms of the model copula may be written:

$\begin{matrix}{{YL} = {\frac{p_{ff} + p_{pf}}{p_{ff} + p_{pf} + p_{fp} + p_{pp}} = {\frac{D_{N}\left( {1,y_{test}} \right)}{D_{N}\left( {1,1} \right)} = {D_{N}\left( {1,y_{test}} \right)}}}} & (77) \\{{EUFF} = {\frac{p_{fp}}{p_{fp} + p_{pp}} = \frac{{D_{N}\left( {x_{use},1} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}{1 - {D_{N}\left( {1,y_{test}} \right)}}}} & (78) \\{{{{OKill}({Mfg})} = {p_{pf} = {{D_{N}\left( {1,y_{test}} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}}}{{{OKill}({Test})} = {\frac{p_{pf}}{p_{ff} + p_{pf}} = \frac{{D_{N}\left( {1,y_{test}} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}{D_{N}\left( {1,y_{test}} \right)}}}} & (80)\end{matrix}$

An example of a perfectly correlated Test and Use is provided. Forperfectly correlated Test and Use the probability density is uniformalong the diagonal of FIG. 22 and C(x, y)=min[x, y]. This in turn means,from Eq. (71), that D(x, y)=min[x, y]. So,

$\begin{matrix}\begin{matrix}{\mspace{79mu} {{D_{N}\left( {x,y} \right)} = {1 - \left\lbrack {1 - {D\left( {x,1} \right)}} \right\rbrack^{N} - \left\lbrack {1 - {D\left( {1,y} \right)}} \right\rbrack^{N} +}}} \\{\left\lbrack {1 - {D\left( {x,1} \right)} - {D\left( {1,y} \right)} + {D\left( {x,y} \right)}} \right\rbrack^{N}} \\{= {1 - \left\lbrack {1 - x} \right\rbrack^{N} - \left\lbrack {1 - y} \right\rbrack^{N} + \left\lbrack {1 - x - y + {\min \left\lbrack {x,y} \right\rbrack}} \right\rbrack^{N}}} \\{= \left\{ \begin{matrix}{1 - \left( {1 - x} \right)^{N}} & {x \leq y} \\{1 - \left( {1 - y} \right)^{N}} & {x \geq y}\end{matrix} \right.}\end{matrix} & (81) \\{\mspace{79mu} \begin{matrix}{{YL} = {{D_{N}\left( {1,y_{test}} \right)} = {1 - \left( {1 - y_{test}} \right)^{N}}}} \\{= {1 - {\exp \left\lbrack {- {N\left( \frac{r_{test}}{\alpha_{test}} \right)}^{\beta}} \right\rbrack}}}\end{matrix}} & (82) \\{\mspace{79mu} \begin{matrix}{{EUFF} = \frac{{D_{N}\left( {x_{use},1} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}{1 - {D_{N}\left( {1,y_{test}} \right)}}} \\{= \left\{ \begin{matrix}\frac{1 - \left( {1 - x_{use}} \right)^{N} - \left\lbrack {1 - \left( {1 - y_{test}} \right)^{N}} \right\rbrack}{1 - \left\lbrack {1 - \left( {1 - y_{test}} \right)^{N}} \right\rbrack} & {x_{use} \geq y_{test}} \\0 & {x_{use} \leq y_{test}}\end{matrix} \right.} \\{= \left\{ \begin{matrix}\frac{\left( {1 - y_{test}} \right)^{N} - \left( {1 - x_{use}} \right)^{N}}{\left( {1 - y_{test}} \right)^{N}} & {x_{use} \geq y_{test}} \\0 & {x_{use} \leq y_{test}}\end{matrix} \right.} \\{= \left\{ \begin{matrix}{1 - {\exp \left\{ {- {N\left\lbrack {\left( \frac{r_{use}}{\alpha_{use}} \right)^{\beta} - \left( \frac{r_{test}}{\alpha_{test}} \right)^{\beta}} \right\rbrack}} \right\}}} & {\frac{r_{use}}{\alpha_{use}} \geq \frac{r_{test}}{\alpha_{test}}} \\0 & {\frac{r_{use}}{\alpha_{use}} \leq \frac{r_{test}}{\alpha_{test}}}\end{matrix} \right.}\end{matrix}} & (83) \\\begin{matrix}{{{OKill}({Mfg})} = {{D_{N}\left( {1,y_{test}} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}} \\{= \left\{ \begin{matrix}0 & {x_{use} \geq y_{test}} \\{1 - \left( {1 - y_{test}} \right)^{N} - \left\lbrack {1 - \left( {1 - x_{use}} \right)^{N}} \right\rbrack} & {x_{use} \leq y_{test}}\end{matrix} \right.} \\{= \left\{ \begin{matrix}0 & {x_{use} \geq y_{test}} \\{\left( {1 - x_{use}} \right)^{N} - \left( {1 - y_{test}} \right)^{N}} & {x_{use} \leq y_{test}}\end{matrix} \right.} \\{= \left\{ \begin{matrix}0 & {\frac{r_{use}}{\alpha_{use}} \geq \frac{r_{test}}{\alpha_{test}}} \\{{\exp \left\{ {- {N\left( \frac{r_{use}}{\alpha_{use}} \right)}^{\beta}} \right\}} - {\exp \left\{ {- {N\left( \frac{r_{test}}{\alpha_{test}} \right)}^{\beta}} \right\}}} & {\frac{r_{use}}{\alpha_{use}} \leq \frac{r_{test}}{\alpha_{test}}}\end{matrix} \right.}\end{matrix} & (84) \\\begin{matrix}{{{OKill}({Test})} = \frac{{D_{N}\left( {1,y_{test}} \right)} - {D_{N}\left( {x_{use},y_{test}} \right)}}{D_{N}\left( {1,y_{test}} \right)}} \\{= \left\{ \begin{matrix}0 & {x_{use} \geq y_{test}} \\\frac{1 - \left( {1 - y_{test}} \right)^{N} - \left\lbrack {1 - \left( {1 - x_{use}} \right)^{N}} \right\rbrack}{1 - \left( {1 - y_{test}} \right)^{N}} & {x_{use} \leq y_{test}}\end{matrix} \right.} \\{= \left\{ \begin{matrix}0 & {\frac{r_{use}}{\alpha_{use}} \geq \frac{r_{test}}{\alpha_{test}}} \\\frac{{\exp \left\{ {- {N\left( \frac{r_{use}}{\alpha_{use}} \right)}^{\beta}} \right\}} - {\exp \left\{ {- {N\left( \frac{r_{test}}{\alpha_{test}} \right)}^{\beta}} \right\}}}{1 - {\exp \left\{ {- {N\left( \frac{r_{test}}{\alpha_{test}} \right)}^{\beta}} \right\}}} & {\frac{r_{use}}{\alpha_{use}} \leq \frac{r_{test}}{\alpha_{test}}}\end{matrix} \right.}\end{matrix} & (85)\end{matrix}$

Eq. (83) shows that for perfect correlation, EUFF vanishes if the Testcondition exceeds the Use condition, and it quantifies the fractionfailing whenever the test condition is made less than the use condition.The behavior of the overkill FOMs complement this, showing the essentialtradeoff between EUFF and overkill. Notice that Use and Test conditionsare quantified by the single parameter r/α and α depends on V_(p),V_(d), and temperature.

Regarding fault tolerance, the array model is generalized to takeaccount of fault tolerance at Test and in Use. At Test fault toleranceis implemented by physically remapping of failed bits to a small numberof rows or columns, whereas in Use fault tolerance is implemented byerror correction redundancy coding (ECC), not physical repair.Statistical models of the effect of fault tolerance schemes on FOMs isdone by expanding the definition of a “good” array to include arrayswith some “bad” bits. “Bad” bits in arrays that are considered “good”are taken to be covered by a fault tolerance scheme. The maximum numberof “bad” bits that can be tolerated is a measure of the repair capacityof the fault tolerance scheme. Only the repair capacity of a faulttolerance scheme is needed to estimate the effect on FOMs. It is notnecessary to know implementation details of the scheme. The expressionsfor FOMs for two fault tolerant cases are derived. The cases are 1) NoRepair at Test, and 2), Repair at Test. In the first of these cases thetester does not actively repair any failing bits that it finds, whereasin the second case the tester can repair some failing bits.

Consider an array made from N of the bits characterized and modeled bythe copula-based model in the DRAM example experiment. The probabilitythat the array has exactly n_(fp) bits in category fp, n_(pf) bits incategory pf, and n_(f) bits in category ff is, by the multinomialtheorem and its Poisson limit,

$\begin{matrix}{{\begin{pmatrix}N \\{n_{pf},n_{fp},n_{ff}}\end{pmatrix}p_{pf}^{n_{pf}}p_{fp}^{n_{fp}}{p_{ff}^{n_{ff}}\left( {1 - p_{pf} - p_{fp} - p_{ff}} \right)}^{N - n_{pf} - n_{fp} - n_{ff}}}\underset{N->\infty}{\rightarrow}{\frac{\lambda_{ff}^{n_{ff}}{\exp \left( {- \lambda_{ff}} \right)}}{n_{ff}!}\frac{\lambda_{pf}^{n_{pf}}{\exp \left( {- \lambda_{pf}} \right)}}{n_{pf}!}\frac{\lambda_{fp}^{n_{fp}}{\exp \left( {- \lambda_{fp}} \right)}}{n_{fp}!}}} & (86)\end{matrix}$

where λ_(pf)=N p_(pf), λ_(fp)=N p_(fp), λ_(ff)=N p_(ff) and wherep_(pf), p_(fp), and p_(ff) are related to the bit-level pseudo-copula Dby Eq. (75) with N=1. It will be shown that expressions for the threeprobabilities P(Passes Test), P(Good in Use), and P(Passes Test and Goodin Use) all involve sums over terms like either side of Eq. (86). FOMsdepend, in turn, on these probabilities via Eqs. (58), (59), (60), and(61). The Poisson limit is well-justified for the DRAM since typicalarrays have many thousands of bits with only tens of failing bits atmost. Moreover, the mathematical manipulations are more tractable inthis limit.

The method of summing sums of terms like Eq. (86) over categories offailure mechanisms (single cell, word-line, etc.) is known. Theinvention described here uses the same method, except that thecategories are pass/fail in Use or Test. It is also known from earlierwork that account can be taken of variation of defect density acrosswafers, lots, and factories by replacing the Poisson terms on the r.h.s.of Eq. (86) by a negative binomial distribution

$\begin{matrix}{{\frac{\Gamma \left( {\alpha + n} \right)}{{n!}{\Gamma (\alpha)}}\frac{\left( {\lambda/\alpha} \right)^{n}}{\left( {1 + {\lambda/\alpha}} \right)^{n + \alpha}}}\underset{\alpha->\infty}{->}\frac{\lambda^{''}{\exp \left( {- \lambda} \right)}}{n!}} & (87)\end{matrix}$

where α quantifies the variation of defect density and α→∞ correspondsto uniform defect density. It is important to note that this descriptionapplies to variations in cell defect density of much greater spatialextent than the size of the N-bit array, so that the defect densitywithin any given array is constant. Ways to extend yield models ofdefect-tolerance to arbitrarily complex chip floor plans and to defectdensity variations, which can occur within the chip have also beenestablished in earlier work. All of these extensions are available tothe method described here, but to minimize clutter the single-mechanismPoisson formulation, Eq. (86), will be used to show the novel aspects ofthe invention. The novel aspects with regard to fault tolerance are 1)calculation of pass/fail probabilities for Test/Use categories, 2)calculation of all important FOMs, not just yield loss, 3) graphicalrepresentation of fault tolerance schemes, 4) efficient ways to computefunctions needed by the theory. These enable earlier work covering otheraspects to be extended to take account of miscorrelation between Testand Use, and to consider FOMs other than yield.

Fault tolerance schemes in Test and in Use may be described by a set ofconstraints on the range of indices over which the sums of terms likeEq. (86) range in expressions for the probability functions appearing inthe expressions for the FOMs, Eqs.(58), (59), (60), and (61). Ingeneral, sets of allowed values of n_(ff), n_(pf), and n_(fp) consistentwith the constraints may be computed once for any test and array designscheme and then be reused to compute FOMs for different values ofλ_(ff), λ_(pf), and λ_(fp). In the examples shown next, attention isconfined to cases for which FOMs may be computed even more convenientlyusing special functions. Two cases will be considered: No Repair atTest, and Repair at Test.

In the No Repair at Test case, Test tolerates, but does not repair,≦n_(t) failing bits, and Use tolerates ≦n_(u) bits. In this case, n, isa measure of the transparency of Test to failing bits. On the otherhand, the Repair at Test case would restore up to n_(t) failing bits tofunctionality by, for example, replacing a word with a bad bit with aword with all good bits. In this case, n_(t) is a measure of the size ofthe supply of spares. n_(u) and _(t) are usually small integers, whichmakes evaluation of various required functions easy. In the following,keep in mind that the first character in labels such as ff, pf, and fprefers to Use, and the second refers to Test.

Turning to the No Repair at Test case, expressions for three probabilityfunctions are needed. The first of these is the probability of PassesTest where an array is defined as passing test with up to n, bitsfailing. The permitted counts of bit categories for arrays in the PassesTest category is the set of integers:

$\begin{matrix}{{PT\_ NR} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}\text{:}}} \\{0 \leq {n_{ff} + n_{pf}} \leq n_{t}} \\{0 \leq n_{fp} < \infty}\end{Bmatrix}} & (88)\end{matrix}$

which leads to

$\begin{matrix}{{{P\left( {{Passes}\mspace{14mu} {Test}} \right)} = {{\sum\limits_{PT\_ R}\; {\frac{\lambda_{ff}^{n_{ff}}^{- \lambda_{ff}}}{n_{ff}!}\frac{\lambda_{fp}^{n_{fp}}^{- \lambda_{fp}}}{n_{fp}!}\frac{\lambda_{pf}^{n_{pf}}^{- \lambda_{pf}}}{n_{pf}!}}} = {R\left( {{\lambda_{ff} + \lambda_{pf}},n_{t}} \right)}}}\mspace{20mu} {where}} & (89) \\{\mspace{79mu} {{R\left( {x,n} \right)} \equiv {^{- x}{\sum\limits_{0 \leq i \leq n}\; {\frac{x^{i}}{i!}.}}}}} & (90)\end{matrix}$

Manipulations leading to Eq. (89) and (90) are shown below. As shownbelow, the function R is related to the cumulative Gamma distribution,which is available in many software function libraries.

The second probability function required for the No Repair at Test caseis the Good in Use probability function where an array is defined asgood in use with up to n_(u) bits failing. The permitted counts of bitcategories for arrays in the Good in Use (irrespective of Test) categoryis the set of integers:

$\begin{matrix}{{GIU\_ NR} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {n_{ff} + n_{fp}} \leq n_{u}} \\{0 \leq n_{pf} < \infty}\end{Bmatrix}} & (91)\end{matrix}$

which, by symmetry w.r.t. Eq.(88), leads to

P(Good in Use)=R(λ_(ff)+λ_(fp) , n _(u)).   (92)

The third probability function required for the No Repair at Test caseis the Passes Test and Good in Use probability function. The permittedcounts of bit categories for arrays in the Passes Test and Good in Usecategory is the set of integers

$\begin{matrix}{{PTGIU\_ NR} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {n_{ff} + n_{pf}} \leq n_{t}} \\{0 \leq {n_{ff} + n_{fp}} \leq n_{u}}\end{Bmatrix}} & (93)\end{matrix}$

which leads to

$\begin{matrix}{{P\left( {{Passes}\mspace{14mu} {Test}\mspace{14mu} {and}\mspace{14mu} {Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)} = {{\sum\limits_{PTGIU\_ NR}\; {\frac{\lambda_{ff}^{n_{ff}}^{- \lambda_{ff}}}{n_{ff}!}\frac{\lambda_{fp}^{n_{fp}}^{- \lambda_{fp}}}{n_{fp}!}\frac{\lambda_{pf}^{n_{pf}}^{- \lambda_{pf}}}{n_{pf}!}}} \equiv {K\left( {\lambda_{ff},\lambda_{fp},n_{u},\lambda_{pf},n_{t}} \right)}}} & (94)\end{matrix}$

Notice that, as expected from the definitions of the probabilityfunctions P(Passes Test), etc., the set of integers PTGIU_NR is theintersection of the sets PT_NR and GIU_NR. The function K, shown in Eq.(146) below, is easy to evaluate because it is a sum of a small finitenumber of terms involving products of the function R.

Sets of integers corresponding to three categories of arrays given byEqs. (88), (91), and (93) may be visualized as regions in bit categoryindex space as shown in FIGS. 23 a-23 c for n_(t)=3 and n_(u)=7. Eachlattice point in this space corresponds to a term like Eq. (86) in theprobability functions given by Eqs. (89), (92), and (94). The magnitudeof the term depends on values of λ_(pf), λ_(fp), and λ_(ff) and islarger for lattice points closer to the origin. The Passes Test, and theGood in Use categories are infinite prisms running down the n_(fp) andn_(pf) axes, respectively. The Passes Test and Good in Use category isthe intersection of these prisms. For no fault tolerance, the allowedintegers collapse to the single point at the origin, (0, 0, 0). Becausethe three probability functions used to compute FOMs correspond toshapes in index space shown in FIGS. 23 a-23 c, the FOMs derived fromthese probability functions via Eqs. (58), (59), (60), and (61) alsocorrespond to volumes in index space. The volume corresponding to YieldLoss is the entire space outside the prism in FIG. 23 a. The volumes forOverkill (Mfg) and EUFF [apart from the normalizing factor, P(PassesTest)] are shown in FIGS. 24 a-24 b. Notice that, for n_(u)>n_(t), thevolume corresponding to Overkill includes array configurations whichwould pass in Use with fairly high probability since the index pointsare close to the origin. That is, as n_(u) increases and n_(u)>n_(t),Overkill becomes larger and more nearly equal to Yield Loss because Testis rejecting more arrays that would have been good in Use. On the otherhand, EUFF is small because all of the terms contributing to it aredistant from the origin. Geometrical considerations like this arehelpful in interpreting the dependencies of the model.

Turning to the Repair at Test case, expressions for the three neededprobability functions are derived. Suppose that Test can repair up ton_(t) bits. In this case, the active intervention of repair at Testchanges the meaning of the array Good in Use criterion involving n_(u).So Eqs. (88), (91), and (93) for the No Repair at Test case arerewritten as described in the following.

The Passes Test probability function for the Repair at Test case isdefined by the same set of integers as the No-Repair at Test casebecause the criterion for rejecting an array at Test depends only on thenumber of bits tolerated at Test, not on whether or not any of thetolerated bits are repaired. So the Passes Test category of arrays inthe Repair at Test case is defined by the set of integers:

$\begin{matrix}{{PT\_ R} = {{PT\_ NR} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {n_{ff} + n_{pf}} \leq n_{t}} \\{0 \leq n_{fp} < \infty}\end{Bmatrix}}} & (95)\end{matrix}$

which leads to the same expression as for the No Repair at Test case:

P(Passes Test)=R(λ_(ff)+λ_(pf), n_(t)).   (96)

Regarding the Good in Use probability function of the Repair at Testcase, if an array is subjected to a repair process at Test with capacityn_(t), then the effective number of ff and pf bits affecting thepost-test classification of the array is the union of the ff and pfcategories, minus n_(t): n_(ff)+n_(pf)−n_(t). If it is assumed that therepair process does not distinguish between ff and pf bits (both kindsare detected as fails in Test), then the proportion of ff and pf bitsaffecting post test array classification is the same as the pre-testproportions of these bit categories. So one may model the post-test ffbit count, which must figure into classification of arrays in Use as

$\begin{matrix}{n_{ff}^{\prime} = {{n_{ff}^{\prime}\left( {n_{ff},n_{pf},n_{t}} \right)} = \left\lceil {{\max \left\lbrack {{n_{ff} + n_{pf} - n_{t}},0} \right\rbrack}\frac{n_{ff}}{n_{ff} + n_{pf}}} \right\rceil}} & (97)\end{matrix}$

where this is rounded up to the next integer. So the set of integersdefining the Good in Use category of arrays for the Repair at Test caseis

$\begin{matrix}{{GIU\_ R} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {{n_{ff}^{\prime}\left( {n_{ff},n_{pf},n_{t}} \right)} + n_{fp}} \leq n_{u}} \\{0 \leq n_{pf} < \infty}\end{Bmatrix}} & (98)\end{matrix}$

Eq. (98) coincides with Eq. (91) when n_(t)=0 becausen_(ff)=n′_(ff)(n_(ff), n_(pf), 0).

In general, n_(ff)≦n′_(ff), but whenever n_(pf) is greater than asufficiently large value, m, then the effect of n_(t) in Eq. (97) isnegligible and n_(ff)=n′_(ff). That is,

n _(ff) =n′ _(ff)(n _(ff) , n _(pf) , n _(t)) for n _(pf) ≧m.   (99)

The geometrical interpretation of m is shown in FIG. 25 b. Careful studyof Eq. (97) shows that m is the smallest integer value (≧0), whichsatisfies

n _(u) =n′ _(ff)(n _(u)+1, m,n _(t))−1   (100)

Eq. (100) is easily solved incrementing m through a small number ofinteger values until the equation is satisfied. So m=m(n_(u), n_(t)) isa function of n_(u), and n_(t). Some special cases are: m(n_(u), 0)=0,m(n, n)=n².

Because of the property n_(ff)≦n′_(ff), the set of integers GIU_NR is asubset of GIU_R. That is

$\begin{matrix}{{{GIU\_ R} = {{GIU\_ NR}\bigcup{\Delta \; {GIU\_ R}}}}{where}} & (101) \\{{\Delta GIU\_ R} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {{n_{ff}^{\prime}\left( {n_{ff},n_{pf},n_{t}} \right)} + n_{fp}} \leq n_{u}} \\{{n_{ff} + n_{fp}} > n_{u}} \\{0 \leq n_{pf} \leq {{m\left( {n_{u},n_{t}} \right)} - 1}}\end{Bmatrix}} & (102)\end{matrix}$

In Eq. (102) the second inequality ensures that the integers in ΔGIU_Rare not in GIU_NR, and the final condition limits the range of n_(pf) tocases where n_(ff)<n′_(ff) is possible, for the purpose of efficiency. Ageometrical interpretation of GIU_R is shown in FIG. 25 b. Notice that,in FIG. 25 b, ΔGIU_R is represented by the 5-sided polyhedron atop theGood-in-Use prism for GIU_NR shown in FIG. 23 b. The sum over theintegers GIU_NR has been given by Eq. (92), to which must be added termsresulting from Eq. (102)

$\begin{matrix}{{{P\left( {{Good}\mspace{14mu} {in}\mspace{14mu} {Use}} \right)} = {{L\left( {\lambda_{ff},\lambda_{fp},\lambda_{pf},n_{u\;},n_{t}} \right)} + {R\left( {{\lambda_{ff} + \lambda_{fp}},n_{u}} \right)}}}\mspace{20mu} {where}} & (103) \\{{L\left( {\lambda_{ff},\lambda_{fp},\lambda_{pf},n_{u\;},n_{t}} \right)} = {\sum\limits_{\Delta \; {GIU\_ R}}\; {\frac{\lambda_{ff}^{n_{ff}}^{- \lambda_{ff}}}{n_{ff}!}\frac{\lambda_{fp}^{n_{fp}}^{- \lambda_{fp}}}{n_{fp}!}\frac{\lambda_{pf}^{n_{pf}}^{- \lambda_{pf}}}{n_{pf}!}}}} & (104)\end{matrix}$

When n_(t)=0 the set ΔGIU_R is empty, then L=0, and the repair-in-Testcase becomes the same as the No-Repair-in-Test case. The function L iseasy to evaluate for the usual case when n_(t) and n_(u) are smallintegers since it is a sum over the small set of integers in the 5-sidedpolyhedron in FIG. 25 b referred to above. Notice that m in the figurelimits the range over which the sum over n_(pf) must be taken.

The Passes Test and Good in Use probability function of the Repair atTest case corresponds to the intersection of allowed indexes for thePasses Test and Good in Use cases previously described:

$\begin{matrix}{{PTGIU\_ NR} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq {{n_{ff}^{\prime}\left( {n_{ff},n_{pf},n_{t}} \right)} + n_{fp}} \leq n_{u}} \\{0 \leq {n_{ff} + n_{pf}} \leq n_{t}}\end{Bmatrix}} & (105)\end{matrix}$

The final inequality in Eq. (105), substituted into Eq. (97), ensuresthat n′_(ff)=0, so Eq. (105) becomes

$\begin{matrix}{{PTGIU\_ R} = \begin{Bmatrix}{n_{ff},n_{pf},{n_{fp}:}} \\{0 \leq n_{fp} \leq n_{u}} \\{0 \leq {n_{ff} + n_{pf}} \leq n_{t}}\end{Bmatrix}} & (106)\end{matrix}$

which has the simple geometrical interpretation shown in FIG. 25 c. Eq.(106) immediately leads to

P(Passes Test and Good in Use)=R(λ_(fp), n_(u))R(λ_(ff)+λ_(pf), n_(t))  (107)

The expressions for the three probabilities, Good in Use, Passes Test,and Passes Test and Good in Use, derived in this section for arraysproduced in a No Repair at Test or Repair at Test case may besubstituted into the expressions for FOMs, Eqs. (58), (59), (60), and(61) to obtain the array FOMs.

Turning now to application of the current invention, environmentalmodels of marginal distributions, wedge copula models, the test coveragemodel, and scaling and fault tolerance models were implemented in anExcel calculator. The calculator makes deterministic calculations ofFOMs from the fitted model. These FOMs, when compared with FOM targets,can be used to determine the Test manufacturing process, datasheetspecification, and fault tolerance requirements of the integratedcircuit memory product. The user interface of the calculator is shown inFIG. 26. Dashed-outlined cells are user inputs, and solid-lined cellsare outputs. The main sections of the interface of the Excel calculatortool are described in more detail in the following.

The Skew, Model Parameters section in FIG. 26 allows the user to selectthe process skew (Table 1). The fitted marginal distributionenvironmental parameters and wedge copula parameters for the selectedskew (Table 3) are displayed. An extra skew called “Spare” which is thesame as the nominal skew but with a perfect correlation copula, M, isavailable in addition to the five skews of Table 1 for which parameterswere extracted from data.

The Memory Architecture section in FIG. 26 allows the user to select thearray size and number of bits tolerated at Test and at Use. The numberof bits tolerated at Test is interpreted in two ways: 1) For No Repairat Test, failing bits detected at Test are tolerated but not repaired.2) For Repair at Test, failing bits detected at Test are tolerated andmade good (repaired). A set of FOMs is generated for each way.

The Test and Use Conditions section in FIG. 26 allows the user to selectretention time and environmental conditions (V_(u), V_(d), T) for Testand for Use. The model also requires specification of the test coveragemodel, which defines how minimum and maximum retention times for a VRTbits are be distributed between Test and Use. The interface provides achoice between the conservative test coverage model in which r_(max)occurs only in Test, and r_(min) occurs only in Use, the aggressive testcoverage model for the opposite assumption, and the symmetrical testcoverage model for which min/max retention times are equally likely tobe in Test or Use.

The Figures of Merit section in FIG. 26 shows the computed four FOMsdefined by Eqs. (58), (59), (60) and (61) for the No-Repair-at-Test, andthe Repair-at-Test cases.

Not shown in FIG. 26 are parts of the interface, which specify targetsand plotting limits for the graphs also produced by the calculator. Thegraphs generated by sweeping the Test retention time past the Userefresh time give a good appreciation of the properties of the model.Because environmental conditions enter the model only through lnα [seeEqs. (62), (63) and (2)], properties of the model may be explored bysetting V_(p), V_(d), and T to a convenient value (the referencecondition in the example shown in FIG. 26), and then sweeping theretention time setting in Test past the Use refresh time specification.Plots are generated for which all the input parameters are entered inthe user interface as described above, except that the Test retentiontime (r_t) is swept between the plotting limits defined for it. Wesuppose a 1 Mb (2²⁰ bits) array is fabricated in the nominal skew, thatthe environmental conditions of both Test and Use are V_(p)=0.45 V,V_(d)=1.2 V, T=125° C., and that the data sheet (Use) refresh time is108 au. The targets for the product are taken as Yield Loss≦20%,Overkill (Mfg)≦2%, and EUFF≦200 DPPM. As the test retention time setpoint is swept from 90 au to 150 au, the Yield Loss, Overkill (Mfg), andEUFF FOMs vary as shown in FIG. 27 to FIG. 31.

It is apparent from FIG. 27 that, without fault tolerance (n_(t)=0,n_(u)=0), it is impossible to find a Test retention time for which YieldLoss, Overkill (Mfg), and EUFF targets are simultaneously met. If thearray can tolerate failing bits by some error-correction mechanism inthe array, then all of the FOMs are reduced and a Test set point existsfor which all targets are satisfied. For the conditions of the example,three is the minimum number of faulty bits that the array must tolerate[n_(t)=3 (No Repair at Test) and n_(u)=3] for a test set point to exist,and FIG. 28 shows FOM characteristics and the range of possible testretention time settings (shaded region) for this case. A set point atthe left edge of this range will minimize Overkill (Mfg) and Yield Losswhile meeting the EUFF target: n_(test)=121 au, YL=4.3%, Overkill(Mfg)=1.2%, EUFF=173 DPPM.

FIGS. 27, 28 and 29 a-29 c show how the shape of the FOM characteristicsreflect the underlying copula model. In the No Repair cases of FIG. 27,FIG. 28, and FIG. 29 a-29 c, vertical asymptotes on a logarithmic plotof the Overkill and EUFF figures of merit correspond to the boundariesof the wedge copula, transformed by the test coverage model, at the Usecondition. If retention time in Test is always r_(max) and in Use it isalways r_(min), (the conservative test coverage model) as in all figuresexcept FIGS. 29 a and 29 b, then the Overkill vertical asymptotecorresponds to the Use condition. The case of the symmetrical testcoverage model is shown in FIG. 29 a and the aggressive test coveragemodel is shown in FIG. 29 b. FIG. 29 c shows how the case of perfectcorrelation makes vertical asymptotes of Overkill and EUFF both alignwith the Use condition. Comparison of FIGS. 27, 29 a, and 29 b gives anexample of how the conservative test coverage model maximizes the modelestimate of EUFF compared to the symmetrical and aggressive testcoverage models.

Another way to implement fault tolerance is to repair arrays at Test.Consider arrays defined as for FIG. 27, except that Test tolerates 2bits but does not repair them. In this case many arrays will fail in Usebecause Test is transparent to arrays with up to 2 bits failing, but Usecannot tolerate any failing bits in the array. This leads to a largevalue of EUFF shown in FIG. 30 a. Now suppose that Test repairs the 2bits that it tolerates. In this case EUFF is much reduced, and FIG. 30 bshows that it will be possible to find a Test set point, which meets thetargets (shaded range).

Finally, consider the case of Repair at Test when Use can tolerate badbits by comparing FIG. 30 b and FIG. 31. The conditions in these figuresare the same, including n_(t)=2, except n_(u)=0 in FIG. 30 b and n_(u)=1in FIG. 31. Increased tolerance of bad bits in Use significantly reducesEUFF in FIG. 31 compared to FIG. 30 b. Tolerance of bad bits in Use alsomakes Overkill the dominant subset of Yield Loss in FIG. 31 because, astolerance of failing bits in Use increases, Test will reject more arraysthat would have been good in Use. Here, it is chosen to determine therange of acceptable set points (shaded) in FIG. 31 only by Yield Lossand EUFF, which determine the cost and quality of the Test. Theincreased Overkill in FIG. 31 is not a “problem”, but is an“opportunity” for some other test method beyond the scope of themanufacturing flow considered here. For example, yield loss rejectscould be screened by a “Use-like” test to recover (some of) the arraysthat are good-in use. Viability of this will be determined by the costof the additional screening.

Turning to a description of a two-part methodology, which integrates thetechniques and methods described thus far; the first part is ModelExtraction shown in FIG. 32 a and FIG. 32 b, and the second part isInference shown in FIG. 33. FIGS. 32 a and 32 b show alternativeapproaches to characterizing sampling variation in the Model Extractionpart of the methodology. While, at a high-level, the two-partmethodology is prior art, the discussion shows how copula-basedstatistical models are integrated into the two-part methodology andimprove key aspects of it. The purpose of the methodology is todetermine the design, test manufacturing and datasheet (end use)specification of an electronic system or integrated circuit product,taking into account dependent attributes of the product. The end resultof Model Extraction is a Statistical Model of the test vehicle fitted tothe data. The Statistical Model is then used by the Inference part ofthe methodology to compute FOMs for product design, manufacturing, anddatasheet specifications different from those of the test vehicle. FOMsof the product of interest are compared with targets, which reflectcorporate manufacturing and quality policies to decide whether thespecifications of the product of interest meet requirements.

The improvements in the methodology afforded by the copula-basedstatistical model have broader applicability than the DRAM example usedto demonstrate them. For example, this methodology may be applied todifferent dependent attributes measured at Test, such as I_(sb)(stand-by current) vs. F_(max), (maximum operating frequency) in orderto optimize test manufacturing screens based on measuring the moreconveniently measured attribute (I_(sb)). The methodology also appliesto integrated circuits, which integrate various types of elements, notjust memory bits, and to electronic systems which integrate componentssuch as integrated circuits and other modules.

Turning now to the Model Extraction aspect of the methodology shown inFIG. 32 a, an Experimental Design is used to acquire test vehicle data(“Real Data”), which is then fitted to a selected model. An ExperimentalDesign is a specification of particular conditions of the experimentsuch as the temperatures, voltages, retention time bins, sample sizesfor each environmental condition and so on, according to one embodimentof the invention. Model Selection includes choosing the form of themarginal distributions (Weibull for the DRAM example, Eq. (1)), theenvironmental model (like Eq. (2) for the DRAM example), and the copulamodel (wedge copula for the DRAM example). Then “Real Data” is fitted tothe selected statistical model that includes the selected marginal,environmental and copula models, to generate “Parameters from Data” likethose in Table 4 of the DRAM example, according to one embodiment of theinvention. An important benefit of the copula-based statistical model isthat fitting of marginal distributions is completely decoupled fromfitting of the copula, so that the marginal and copula model-fittingsteps may be done in any sequence. To characterize the statisticaluncertainty inherent in the Experimental Design and details of theParameter Extraction methods (e.g. least-squares, sample tau, etc.), thecopula-based model parameter set fitted to Real Data is used tosynthesize multiple Data Replicates of the entire dataset acquired usingthe Experimental Design, as shown in FIG. 32 a.

Parameters (e.g. lnα₀, Q, a, b, c for the DRAM) are extracted from eachData Replicate to produce a Parameter Replicate. The variation acrossthe Parameter Replicates characterizes the sampling variation of theexperiment and the Parameter Extraction methods. A key enabler of theMonte-Carlo synthesis of Data Replicates to characterize statisticalvariation is the use of a geometrical copula. This is because adequatecomputational efficiency is only feasible for the highly data-censoredtest application if Monte-Carlo data synthesis can be confined to theregion accessed by the experiment. This is possible for geometricalcopulas, but generally not for Archimedian, Elliptical (Gaussian andt-copulas), Marshall-Olkin and most, if not all, other types of copula.It is also not feasible for the conventional multi-normal (non-copula)modeling approach. A key benefit of using Monte-Carlo synthesized DataReplicates is that a different Experimental Design from the one used togenerate the test vehicle data (“Real Data”) may be assumed in thesynthesis, enabling optimization of the Experimental Design andParameter Extraction methods for subsequent data acquisition andmodel-fitting. FIG. 32 b shows how data resampling methods other thanMonte-Carlo synthesis, for example the Bootstrap method, may also beused to derive Parameter Replicates to characterize sampling variationand Parameter Extraction methods. However, data resampling methods donot admit the possibility of varying the Experimental Design orParameter Extraction methods as does the geometrical-copula-enabledMonte-Carlo synthesis method.

Further guidance for the Experimental Design and Model Selectionelements of the Model Extraction part of the methodology of FIGS. 32 aand 32 b is provided next. This is important because the experimentaldesign for test vehicle data acquisition and model extraction ultimatelydetermine how aggressive test set points and datasheet specificationsmay be.

Guidance for the Experimental Design element of Model Extraction inFIGS. 32 a and 32 b is:

-   -   Acquire data at a number of environmental conditions spanning        the datasheet specification and test conditions of the product.        Test conditions, environmental conditions and sample sizes        should be chosen to produce some failures at all test        conditions, and a significant number of failures (at least 100s)        at the most-stressed corners.    -   Acquire data using a test vehicle designed so that it is        possible to measure attributes separately on modules from which        the product chip will be constructed. In the DRAM example,        bit-level data was acquired, so that models for hypothetical        arrays of any size could be derived.    -   Acquire “continue-on-fail” measured values of all attributes of        interest on each measured module across the entire range of        environmental and test conditions. Do not terminate measurements        because the attribute values exceed some specification limit.        Binned data, as in the DRAM experiment, can also be used but is        less preferred.    -   Sample sizes can be determined by using the copula-based        statistical model with a “guessed” set of parameters often based        on similar products and technologies. Larger sample sizes reduce        the “guard bands” (discussed below) necessary to contain risks        of sample variation in the experiment.    -   Reduce data requirements for the experimental design by        regarding some parameters as “known” and set them to        conservative values. For example, if the underlying mechanism is        regarded as known, then acceleration parameters, such as Q in        the DRAM example, may be set to “literature” values. Another way        to reduce the data requirements experimental design is to        characterize only a small sample across the full span of        environmental conditions, but characterize a much larger sample        at the “center” environmental condition.

Guidance for the Model Selection element of Model Extraction in FIGS. 32a and 32 b is:

-   -   Marginal distributions should be chosen to have flexibility in        scale and shape. Weibull distributions are often a good choice        because the shape parameter can control the rate of increase of        an attribute with stress.    -   Environmental dependence of marginal model parameters,        preferably just the scale parameter, is fitted to models guided        by physics of expected mechanisms. For example, Arrhenius        dependence of a on temperature is a good choice for thermal        effects.    -   Choose a copula that “looks like” the data. This can be        determined by synthesizing empirical copula data and comparing        it with the real empirical data. Because Test data is so heavily        censored it is desirable to choose a copula that can efficiently        synthesize data targeting the restricted range of the data.    -   If possible, use copulas with an obvious geometrical        interpretation. If necessary, construct one. Avoid uncritical        application of copulas for which the behavior beyond the scope        of the data may introduce artifacts. Geometrical copulas are        recommended because of the ease of construction and        interpretation.    -   Choose a copula with parametric range that can span a range of        subpopulation tau covering observed values of tau. For the DRAM        example the Gaussian copula only marginally meets this        requirement because the parameter, p, was pushed very close to        its limit in order to fit the data.    -   Minimize the number of copula model parameters. If possible,        choose a copula with one parameter so that it can be determined        by a measurement of Kendall's tau. If necessary to add        parameters, do so by constructing linear combinations of easily        interpretable fundamental (e.g. Frechet upper bound) and        geometrical copulas.    -   Tail dependence of the copula should align with expected        intrinsic and defect behavior. For example, the Gaussian copula        by itself (not in linear combination with M) has LT=0, and so is        not a good candidate for the DRAM data.    -   If possible, choose a copula for which the copula parameter(s)        do not vary with environmental condition, or censoring. If an        environmentally independent copula model can be found, as in the        case of the DRAM example, then set points and datasheet        specifications with given values of r_(test)/α_(test) and        r_(use)/α_(use), respectively, give the same FOMs. This “set        point equivalency” provides flexibility in integrating tests        into a larger suite of tests and in setting datasheet        requirements. For the DRAM example, the diagonal stripe copula        violated this guideline because the best fit of the parameter d,        depends on the environmental condition, but the wedge copula and        the Gaussian copula met (less so) this guideline.    -   Choose a copula for which a censored subpopulation corresponding        to the data can be synthesized with complete efficiency.        Geometrical copulas satisfy this requirement.

Turning to the Inference aspect of the methodology shown in FIG. 33,when the Model Extraction aspect of the methodology shown in FIGS. 32 aand 32 b is complete, the parameter sets extracted from the test vehicledata (“Parameters from Data”) and Parameter Replicates derived byresampling or by Monte-Carlo synthesis may be used to do “what-if'studies to optimize the design, test manufacturing and datasheet (enduse) specification of an integrated circuit product different from thetest-vehicle.

Regarding Product Definition in the Inference aspect of the methodologyshown in FIG. 33, the model fitted to the test vehicle module-level data(bit-level for the DRAM example) must be scaled to the full size of theproduct, and fault tolerance features and the test coverage model of theproduct must be specified. These aspects are set early in the productlifecycle when the product design is fixed.

Regarding Scenario Definition in the Inference aspect of the methodologyof FIG. 33, a scenario defines Test and Use (datasheet) conditions(supply voltages, temperatures, refresh times, etc.) for the product.These aspects can be adjusted late in the product lifecycle whendatasheets for the product and its variants are published to customers.Test conditions can be set at any time, and are frequently adjustedduring manufacturing (that is, very late) to optimize manufacturingfigures of merit.

Regarding Policy in the Inference aspect of the methodology of FIG. 33,figure of merit targets are set at the highest levels of corporatepolicy. Targets for producer-oriented manufacturing FOMs such as yieldloss and overkill are determined by financial and marketing cost modelsas shown earlier in the description of the invention. Targets forcustomer-oriented quality FOMs such as end-use-fail-fraction (EUFF) andreliability indicators are determined by competitive and marketingconsiderations. Confidence limits are driven by the costs of theexperimental design and data acquisition required to build more precisetest vehicle models, versus the manufacturing costs due to yield lossand overkill associated with the Test and datasheet guard bands requiredfor less precise test vehicle models.

Regarding the Confidence Limits aspect of Policy shown in FIG. 33, Testand Use settings must be “guard-banded” to control risks due to thestatistical variation of the experimental design and parameterextraction methods used in the Model Extraction aspect (FIGS. 32 a and32 b) of the methodology. This variation is characterized by theParameter Replicates from synthesized or resampled data shown in FIGS.32 a and 32 b.

The following is an example showing how Test and Use set points areguard-banded according to one embodiment. In FIG. 28 the optimum setpoint would appear to be at the left edge of the shaded zone ofacceptable set points, because the yield loss and overkill is minimizedthere while EUFF just meets the target. This set point, computed using“Parameters from Data” directly extracted from test vehicle data (“RealData”) in FIG. 32 a or 32 b, is called the “nominal” set point. It isapparent from FIG. 28 that the EUFF is rapidly varying at the nominalset point, so a small variation in underlying model parameters couldlead to unacceptably large EUFF. This risk can be contained by using theParameter Replicates extracted from synthesized or resampled testvehicle data to calculate an envelope of Figure of Merit Distributionsaround the FOM corresponding to the nominal set point, as shown in FIG.33. This envelope is then used to shift the nominal set point so thatthe FOMs at this “guard-banded” set point give a probability overlap ofthe targets meeting policy-determined confidence limits. FIG. 34 is aschematic drawing showing this method, according to one embodiment.Overkill is not shown, for simplicity. The semi-analytic deterministiccalculation of FOMs from model parameters and Product Definition andScenario Definition made possible by the copula-based model of theproduct and test manufacturing process makes this method feasiblebecause the calculation of FOMs for each Parameter Replicate is nearlyinstantaneous. Previously, for each Parameter Replicate, a Monte-Carlo(MC) synthesis of individual units of product passing through Test andUse was done and counts of various categories of Pass/Fail in Test/Usewere accumulated to estimate FOMs. Monte-Carlo-estimation of FOMs foreach of the Parameter Replicates is computationally extremelycumbersome, if not unfeasible because of the large MC sample sizesrequired to estimate FOMs to sufficient precision. It must be emphasizedthat the guard bands computed from synthesized or resampled DataReplicates account only for statistical sampling error in theexperimental design and model-fitting methods. They do not account forerrors in the selection of models.

Regarding estimation of model-selection sensitivities, thesemi-analytic, deterministic, and modular nature of the copula-basedstatistical model facilitates the estimation since different modelcomponents may be changed without disturbing other parts of the model,and calculation of FOMs is deterministic and so is virtuallyinstantaneous. The ability of the Excel tool for the DRAM example to trydifferent copula models and test coverage models and instantaneouslycompute FOMs is an embodiment of this feature of the invention.

Turning to derivations of important mathematical results used in thepreceding description of the invention, the first topic is Monte-Carlosynthesis from geometrical copulas. The strategy is to sample from theregions of uniform probability density in the pseudo-copula A used tostart the construction of the copula, and map them to the copula usinginverses of the marginal cdfs of the pseudo-copula. A key aspect of theinvention is that data can be synthesized from a subspace of ageometrical copula with perfect efficiency, that is, without rejectingany sampled points. This is useful for integrated circuit testmanufacturing applications since only the points near the origin of thecopula are of interest, so needless sampling over the entire space ofthe copula can be avoided. This is shown next for regions near theorigin of the stripe and wedge copulas, which are two embodiments of theinvention. The same can be done for any region of any geometrical copulaconstructed by the same method.

The efficient method of Monte-Carlo sampling for a geometrical copula isbased on the fact that any parallelogram containing an area of uniformprobability density can be filled with uniformly distributed randompoints by weighting two basis vectors which span the parallelogram eachwith a random number sampled independently from the uniform distributionon [0, 1]. Every sampled point will lie within the parallelogram.Triangular areas of uniform probability density may similarly be coveredwith a uniform density of random points by considering a parallelogramconstructed from the triangle and its reflection, and reflecting anysampled point falling in the “wrong” half of the parallelogram into thetriangle of interest.

It will be recalled that construction of the stripe copula begins with apseudo-copula, shown in FIG. 13, in which a diagonal stripe of uniformprobability density is drawn across the diagonal. Derivation of thesampling algorithm begins by decomposing the uniform stripe of thestripe pseudo-copula bounded by a region [0, a]² into triangles andrectangles which must be covered by a random density of points. Twocases may be identified. In FIG. 35 a, Case A, merely requires that asmall square be filled with uniform random points. In FIG. 35 b, Case B,divides the region to be filled into two halves of a square, regions 1and 2, separated by a rectangle, 3. The procedure for Case B is asfollows: A uniformly distributed random number, u₁, is generated todecide whether to place the point in 3, or in the divided square, 1 and2. This decision is based on the area ratio of rectangle 3 versus thedivided square, 1 and 2. If the point goes into 3, two uniformlydistributed random numbers u₂ and u₃ are used to place a point in 3.This is done using the basis vectors, which span 3. On the other hand,if the point is to be placed in the divided square, 1 and 2, the pointis placed in a d×d square, but if u₂+exceeds unity the point isdisplaced by (a, a) so that it falls into the triangle 2.

So, the detailed algorithm to generate samples from the region [0,a]² ofthe stripe pseudo-copula, and therefore region [0,f⁻¹(a)]² of the stripecopula, where f⁻¹ is given by Eqs. (31) and (32), is

Case A. 0≦a≦d. Generate two random numbers from the uniform distributionon [0, 1], u₁, u₂ and place a point at

u=au₁, v=au2   (108)

Case B. d≦a≦1.

Generate three random numbers from the uniform distribution on [0, 1],u₁, u₂ and u₃.

If 0≦u₁≦d/(2a−d) and if u₂+u₃<1 place a point at (FIG. 35 b, region 1)

u=du₂, v=du₃   (109)

If 0≦u₁≦d/(2a−d) and if u₂+u₃≧1 place a point at (FIG. 35 b, region 2)

u=a+d(u ₂−1), v=a+d(u ₃−1)   (110)

If d/(2a−d)≦u₁≦1 then place a point at (FIG. 35 b, region 3)

u=d(1−u ₂)+(a−d)u ₃

v=du ₂+(a−d)u ₃   (111)

The generated point of the stripe pseudo-copula is mapped to the stripecopula by

x=f ⁻¹(u) y=f⁻¹(v)   (112)

where f⁻¹ is given by. Eqs. (31) and (32).

Use has been made of the fact that the sum of areas 1 and 2 in the FIG.35 b is a fraction d/(2a−d) of the sum of areas 1, 2, and 3 in thefigure. To derive Eq. (111) note that e₁ and e₂ span region 3 in case B,and the point [d, 0] is õ,

{tilde over (e)} ₁=(−î+ĵ)d

{tilde over (e)} ₂=(î+ĵ)(a−d)

õ=îd   (113)

where î and ĵ are orthogonal unit vectors spanning [0, 1]². So forregion 3, a point is placed at

{tilde over (r)}=õ+{tilde over (e)} ₁ u ₂ +{tilde over (e)} ₂ u ₃=(d−du₂+(a−d)u ₃)î+(u ₂ d+(a−d)u ₃)ĵ  (114)

from which Eq. (111) follows.

Derivation of the sampling algorithm for the wedge copula starts fromthe wedge pseudo-copula shown in FIG. 15, also shown with additionalnotations in FIG. 36. Consider first the problem of filling the regionsI and II in FIG. 36 with uniformly distributed random points. To place arandom point in triangle I in FIG. 36 in (u, v) space, sample u₁ and u₂independently from the uniform distribution on [0, 1], rejecting thesample if u₁>u₂.

$\begin{matrix}\begin{matrix}{\overset{\rightarrow}{r} = {{u_{1}{\overset{\rightarrow}{e}}_{1}} + {u_{2}{\overset{\rightarrow}{e}}_{2}}}} \\{= {{\left\lbrack {u_{1} + {\left( {u_{2} - u_{1}} \right)/c}} \right\rbrack \hat{i}} + {u_{2}\hat{j}}}}\end{matrix} & (115)\end{matrix}$

In Eq. (115) ē₁ and ē₂ are basis vectors spanning triangle I as shown inFIG. 36. Decomposition into the orthogonal unit vectors spanning theunit square in FIG. 36 gives the second equation. By symmetry, to placea random point in triangle II in FIG. 36, reject the sample if u₂>u₁,and place the point at

r=u₁ î+[u ₂+(u ₁ −u ₂)/c]ĵ  (116)

This may be generalized to sample the subpopulation of the wedge copula[0, a]² which corresponds to the sub area [0, f(a)]² of thepseudo-copula shown in FIG. 36. To do this, the uniform random variablesu₁ and u₂ described in the preceding discussion should be independentlysampled from [0, f(a)], rather than [0, 1]².

So the algorithm to sample a region [0, a]² of the wedge copula is:

-   -   1. Sample two independent uniform numbers, u₁ and u₂ from a        uniform distribution on [0, f(a)] (a=1 samples the entire        copula).    -   2. If u₂≦u₁ then place a point at

$\begin{matrix}{{x = {f^{- 1}\left( {u_{1} + \frac{u_{2} - u_{1}}{c}} \right)}}{y = {f^{- 1}\left( u_{2} \right)}}} & (117)\end{matrix}$

-   -   3. Else if u₂<u₁ then place a point at

$\begin{matrix}{{x = {f^{- 1}\left( u_{1} \right)}}{y = {f^{- 1}\left( {u_{2} + \frac{u_{1} - u_{2}}{c}} \right)}}} & (118)\end{matrix}$

where f⁻¹ is given by Eq. (43).

Regarding the subpopulation tau, an expression was given in Eq. (25) fortau of a subpopulation of a copula in the region J²=[0, u_(a)]×[0,v_(b)]. This is needed so that the parameter of a single-parametercopula can be obtained from the value of tau calculated from censoreddata. To derive Eq. (25), a formula for the copula of the subpopulationis substituted into the formula for tau, Eq. (23). The probabilitydensity function for the region J² is

$\begin{matrix}{{D^{\prime}\left( {u,v} \right)} = \frac{C\left( {u,v} \right)}{C\left( {u_{a},v_{b}} \right)}} & (119)\end{matrix}$

This may be converted into a copula by using the marginal distributionfunctions

$\begin{matrix}{u^{\prime} = {\frac{C\left( {u,v_{b}} \right)}{C\left( {u_{a},v_{b}} \right)} = {f(u)}}} & (120)\end{matrix}$

and similarly, v′=C(u_(a), v)/C(u_(a), v_(b))=g(v). Notice that theseare not uniform, f(u)≈u, and g(v)≈v, because u_(a)≈1 and v_(b)≈1. Thecopula for the region J², in terms of the transformed variables, is

$\begin{matrix}{{D\left( {u^{\prime},v^{\prime}} \right)} = {\frac{C\left( {u,v} \right)}{C\left( {u_{a},v_{b}} \right)} = {{\frac{C\left( {{f^{- 1}\left( u^{\prime} \right)},{g^{- 1}\left( v^{\prime} \right)}} \right)}{C\left( {u_{a},v_{b}} \right)}\left\lbrack {u^{\prime},v^{\prime}} \right\rbrack} \in \left\lbrack {0,1} \right\rbrack^{2}}}} & (121)\end{matrix}$

So, using Eq. (23), the subpopulation tau is

$\begin{matrix}\begin{matrix}{\tau_{Subpopulation} = {{4{\int_{0}^{1}{\int_{0}^{1}{{D\left( {u^{\prime},v^{\prime}} \right)}\frac{\partial{D\left( {u^{\prime},v^{\prime}} \right)}}{{\partial u^{\prime}}{\partial v^{\prime}}}\ {u^{\prime}}\ {v^{\prime}}}}}} - 1}} \\{= {{4{\int_{0}^{1}{\int_{0}^{1}{{D\left( {u^{\prime},v^{\prime}} \right)}\frac{\partial{D\left( {u^{\prime},v^{\prime}} \right)}}{{\partial u}{\partial v}}\frac{\partial u}{\partial u^{\prime}}\frac{\partial v}{\partial v^{\prime}}\ {u^{\prime}}{v^{\prime}}}}}} - 1}} \\{= {{4{\int_{0}^{u_{a}}{\int_{0}^{v_{b}}{{D\left( {u^{\prime},v^{\prime}} \right)}\frac{\partial{D\left( {u^{\prime},v^{\prime}} \right)}}{{\partial u}{\partial v}}\ {u}\ {v}}}}} - 1}} \\{= {{\frac{4}{C^{2}\left( {u_{a},v_{b}} \right)}{\int_{0}^{u_{a}}{\int_{0}^{v_{b}}{{C\left( {u,v} \right)}\frac{\partial{C\left( {u,v} \right)}}{{\partial u}{\partial v}}\ {u}\ {v}}}}} - 1}}\end{matrix} & (122)\end{matrix}$

which is the desired expression, Eq. (25).

Since geometrical copulas for which the subpopulation tau given by Eq.(122) is independent of the degree of censoring are particularly useful,it is convenient to have a sufficient condition for censor-independenceof subpopulation tau by which copulas with this property may beidentified. Suppose that a copula is expressed as

C(x, y)=A[f(x),f(y)]  (123)

where A is a pseudo-copula satisfying, for a≦1,

A(a×u, a×v)=a² A(u, v)   (124)

The subpopulation tau of C is

$\begin{matrix}\begin{matrix}{\tau_{Subpopulation} = {{\frac{4}{C^{2}\left( {\alpha,\alpha} \right)}{\int_{0}^{\alpha}\ {{u}{\int_{0}^{\alpha}{{{{vC}\left( {u,v} \right)}}\frac{\partial{C\left( {u,v} \right)}}{{\partial u}{\partial v}}}}}}} - 1}} \\{= {\frac{4}{A^{2}\left\lbrack {{f(\alpha)},{f(\alpha)}} \right\rbrack}{\int_{0}^{\alpha}\ {{u}{\int_{0}^{\alpha}\ {{{vA}\left\lbrack {{f(u)},{f(v)}} \right\rbrack}}}}}}} \\{{\frac{\partial{A\left\lbrack {{f(u)},{f(v)}} \right\rbrack}}{{\partial u}{\partial v}} - 1}} \\{= {\frac{4}{{f^{4}(\alpha)}{A\left( {1,1} \right)}}{\int_{0}^{f{(\alpha)}}\ {{x}{\int_{0}^{f{(\alpha)}}\ {{y}\frac{{\partial u}{\partial v}}{{\partial x}{\partial y}}}}}}}} \\{{{A\left( {x,y} \right)\frac{\partial{A\left( {x,y} \right)}}{{\partial u}{\partial v}}} - 1}} \\{= {{\frac{4}{f^{4}(\alpha)}{\int_{0}^{f{(\alpha)}}\ {{x}{\int_{0}^{f{(\alpha)}}\ {{{{yA}\left( {x,y} \right)}}\frac{\partial{A\left( {x,y} \right)}}{{\partial x}{\partial y}}}}}}} - 1}}\end{matrix} & (125)\end{matrix}$

Now set x=f(a)×x′ and x=f(a)×y′, so

$\begin{matrix}\begin{matrix}{\tau_{Subpopulation} = {\frac{4}{f^{4}(\alpha)}{\int_{0}^{1}{{f(\alpha)}\ {x^{\prime}}{\int_{0}^{1}{{f(\alpha)}\ {y^{\prime}}{A\left\lbrack \left( {{{f(\alpha)}x^{\prime}},{{f(\alpha)}y^{\prime}}} \right) \right\rbrack}}}}}}} \\{{{\frac{\partial{A\left\lbrack {{{f(\alpha)}x^{\prime}},{{f(\alpha)}y^{\prime}}} \right\rbrack}}{{\partial x^{\prime}}{\partial y^{\prime}}}\frac{1}{f^{2}(\alpha)}} - 1}} \\{= {\frac{4}{f^{4}(\alpha)}{\int_{0}^{1}{{f(\alpha)}\ {x^{\prime}}{\int_{0}^{1}{{f(\alpha)}\ {y^{\prime}}{f^{2}(\alpha)}{A\left( {x^{\prime},y^{\prime}} \right)}}}}}}} \\{{{\frac{{f^{2}(\alpha)}{\partial{A\left( {x^{\prime},y^{\prime}} \right)}}}{{\partial x^{\prime}}{\partial y^{\prime}}}\frac{1}{f^{2}(\alpha)}} - 1}} \\{= {{4{\int_{0}^{1}\ {{x^{\prime}}{\int_{0}^{1}\ {{y^{\prime}}{A\left( {x^{\prime},y^{\prime}} \right)}\frac{\partial{A\left( {x^{\prime},y^{\prime}} \right)}}{{\partial x^{\prime}}{\partial y^{\prime}}}}}}}} - 1}}\end{matrix} & (126)\end{matrix}$

which is independent of the degree of censoring, QED. Moreover, thesubpopulation tau for any degree of censoring is the same as thepopulation tau.

In the particular case of the wedge copula, an expression for Kendall'stau for the entire population's copula may be computed analytically, aswill be shown. Moreover, because the wedge copula satisfies thecondition of censor-independence of the sub-population tau, thisexpression is also true for a subpopulation like J². For the wedgecopula,

$\begin{matrix}{{\tau (c)} = {{{4{\int_{0}^{1}{{x}{\int_{0}^{1}{{{{yWe}\left( {x,{y;c}} \right)}}\frac{\partial^{2}{{We}\left( {x,{y;c}} \right)}}{{\partial x}{\partial y}}}}}}} - 1} = {{4I} - 1}}} & (127)\end{matrix}$

Since

We(x, y)=A(u, v)

u=f(x), v=f(y)   (128)

Eq. (127) becomes

$\begin{matrix}\begin{matrix}{I = {\int_{0}^{1}\ {{x}{\int_{0}^{1}\ {{{{yWe}\left( {x,y} \right)}}\frac{\partial^{2}{{We}\left( {x,y} \right)}}{{\partial x}{\partial y}}}}}}} \\{= {\int_{0}^{1}{{u}\frac{x}{u}{\int_{0}^{1}{{v}\frac{y}{v}{A\left( {u,v} \right)}\frac{\partial^{2}{A\left( {u,v} \right)}}{{\partial u}{\partial v}}\frac{u}{x}\frac{v}{y}}}}}} \\{{= {\int_{0}^{1}{{u}{\int_{0}^{1}{{{{vA}\left( {u,v} \right)}}\frac{\partial^{2}{A\left( {u,v} \right)}}{{\partial u}{\partial v}}}}}}},}\end{matrix} & (129)\end{matrix}$

so the evaluation of tau can be done entirely in terms of the pseudocopula A. Evaluation of the integral is facilitated by observing; 1) Thesecond mixed derivative of A(u, v) is the pdf of A, which is a constantequal to c/(c−1) inside the wedge and zero elsewhere. 2) By symmetryacross the diagonal, the desired integral is twice the integral of thelightly shaded zone in FIG. 37. 3) Since A(u, v) vanishes outside thewedge, the v-integration limits may be changed so that for each u, vranges from u/c to u as shown by the small dark stripe in FIG. 37. Usingthese observations,

$\begin{matrix}{I = {2{\int_{0}^{1}{{u}{\int_{u/c}^{u}\ {{{{vA}\left( {u,v} \right)}}{\frac{c}{c - 1}.}}}}}}} & (130)\end{matrix}$

Inside the wedge, where the argument of Eq. (130) is evaluated, usingEq. (41), the pseudo-copula may be written

$\begin{matrix}{{A\left( {u,v} \right)} = {\frac{c}{c - 1}{\left( {{uv} - \frac{u^{2}}{2c} - \frac{v^{2}}{2c}} \right).}}} & (131)\end{matrix}$

So, Eq. (130) becomes

$\begin{matrix}{{I = {{2{\int_{0}^{1}\ {{u}{\int_{u/c}^{u}\ {{{v\left( \frac{c}{c - 1} \right)}^{2}}\left( {{uv} - {\frac{1}{2c}\left( {u^{2} + v^{2}} \right)}} \right)}}}}} = {2\left( \frac{c}{c - 1} \right)^{2}\left( {I_{a} + I_{b} + I_{c}} \right)}}}\mspace{20mu} {where}} & (132) \\{I_{a} = {{\int_{0}^{1}\ {{u}{\int_{u/c}^{u}\ {{vuv}}}}} = {{{\int_{0}^{1}\ {{{uu}} \times \frac{1}{2}v^{2}}}|_{u/c}^{u}} = {{\frac{1}{2}\left( {1 - \frac{1}{c^{2}}} \right){\int_{0}^{1}\ {{uu}^{3}}}} = {\frac{1}{8}\left( {1 - \frac{1}{c^{2}}} \right)}}}}} & (133) \\{I_{b} = {{{- \frac{1}{2c}}{\int_{0}^{1}\ {{u}{\int_{u/c}^{c}\ {{vu}^{2}}}}}} = {{{{- \frac{1}{2c}}{\int_{0}^{1}{u^{2}{u} \times v}}}|_{u/c}^{u}} = {{{- \frac{1}{2c}}\left( {1 - \frac{1}{c}} \right){\int_{0}^{1}{u^{3}{u}}}} = {{- \frac{1}{8c}}\left( {1 - \frac{1}{c}} \right)}}}}} & (134) \\{I_{c} = {{{- \frac{1}{2c}}{\int_{0}^{1}\ {{u}{\int_{u/c}^{u}\ {{vv}^{2}}}}}} = {{{- \frac{1}{2c}}{\int_{0}^{1}\ {{u\left( {{\frac{1}{3}v^{3}}|_{u/c}^{u}} \right)}}}} = {{{- \frac{1}{6c}}\left( {1 - \frac{1}{c^{3}}} \right){\int_{0}^{1}\ {{uu}^{3}}}} = {{- \frac{1}{24c}}\left( {1 - \frac{1}{c^{3}}} \right)}}}}} & (135)\end{matrix}$

Summing these,

$\begin{matrix}\begin{matrix}{{I_{a} + I_{b} + I_{c}} = {{\frac{1}{8}\left( {1 - \frac{1}{c^{2}}} \right)} - {\frac{1}{8c}\left( {1 - \frac{1}{c}} \right)} - {\frac{1}{24c}\left( {1 - \frac{1}{c^{3}}} \right)}}} \\{= {\frac{1}{8c^{4}}\left( {c^{4} - - c^{3} + - {\frac{1}{3}c^{3}} + \frac{1}{3}} \right)}} \\{= {\frac{1}{24c^{4}}\left( {{3c^{4}} - {4c^{3}} + 1} \right)}} \\{= \frac{\left( {c - 1} \right)^{2}\left( {{3c^{2}} + {2c} + 1} \right)}{24c^{4}}}\end{matrix} & (136)\end{matrix}$

and using Eq. (132)

$\begin{matrix}{I = \frac{{3c^{2}} + {2c} + 1}{12c^{2}}} & (137)\end{matrix}$

Substitution of Eqs. (137) into Eq. (127) gives the convenient formula

$\begin{matrix}{\tau_{Subpopulation} = {\frac{{2c} + 1}{3c^{2}}.}} & (138)\end{matrix}$

which, when inverted, may be used to infer a value of c from a value oftau derived from a subpopulation.

Turning now to functions arising in fault-tolerant figures of merit, thefunctions R and K appearing in Eq. (89), Eq. (92), and Eq. (94),respectively, arise when terms like Eq. (86) are summed over sets ofintegers defined for probability functions arising in the No Repair atTest case.

The function R arises for the Passes Test or Good in Use probabilityfunctions of the No Repair at Test case. In either case a set ofintegers is defined as follows:

R={n ₁ , n ₂ , n ₃:(2≦n ₁ +n ₂ ≦n)

(0≦n ₃≦∞)

(n ₁ , n ₂₃ , n ₃∈

⁾)}  (139)

The integers n_(i) and n₂ in this set can be generated by

n ₁ =k−l, n ₂ =l, where 0≦l≦k,0≦k≦n   (140)

so the sum of terms such as Eq. (86) is

$\begin{matrix}{\begin{matrix}{{\sum\limits_{R}\; {\frac{\lambda_{1}^{n_{1}}^{- \lambda_{1}}}{n_{1}!}\frac{\lambda_{2}^{n_{2}}^{- \lambda_{2}}}{n_{2}!}\frac{\lambda_{3}^{n_{3}}^{- \lambda_{3}}}{n_{3}!}}} = {\sum\limits_{\substack{{k = 0},n \\ {l = 0},k}}\; {\frac{\lambda_{1}^{k - l}^{- \lambda_{1}}}{\left( {k - l} \right)!}\frac{\lambda_{2}^{l}^{- \lambda_{2}}}{l!}}}} \\{\left( {{\sum\limits_{{n_{3} = 0},\infty}\; \frac{\lambda_{3}^{n_{3}}^{- \lambda_{3}}}{n_{3}!}} = 1} \right)} \\{= {{\exp \left\lbrack {- \left( {\lambda_{1} + \lambda_{2}} \right)} \right\rbrack}{\sum\limits_{{k = 0},T}\; \frac{1}{k!}}}} \\{\left( {\sum\limits_{{l = 0},k}\; {\frac{k!}{{\left( {k - l} \right)!}{l!}}\lambda_{1}^{k - l}\lambda_{2}^{l}}} \right)} \\{= {{\exp \left\lbrack {- \left( {\lambda_{1} + \lambda_{2}} \right)} \right\rbrack}{\sum\limits_{{k = 0},n}\; \frac{\left( {\lambda_{1} + \lambda_{2}} \right)^{k}}{k!}}}} \\{= {R\left( {{\lambda_{1} + \lambda_{2}},n} \right)}}\end{matrix}\mspace{20mu} {where}} & (141) \\{\mspace{79mu} {{R\left( {x,n} \right)} \equiv \left\{ \begin{matrix}{^{- x}{\sum\limits_{{k = 0},n}\; \frac{x^{k}}{k!}}} & {n \geq 0} \\0 & {n < 0.}\end{matrix} \right.}} & (142)\end{matrix}$

The function R vanishes when n is a negative integer since R is null inthis case. Notice that

R(x, 0)=e ^(−x)

R(x, ∞)=1   (143)

R is related to the cumulative Erlang and Gamma distributions and so isavailable in function libraries such as Excel, for example: R(x,n)=1−GAMMADIST(x, n+1, TRUE) (n≧0).

The function K in Eq. (94) arises for the Passes Test and Good in Useprobability function of the No Repair at Test case. K is defined by theset of integers

K={n ₁ , n ₂ , n ₃:(0≦n ₁ +n ₂ ≦m)

(0≦₁ +n ₃ ≦n)

(n ₁ , n ₂ , n ₃∈

⁾)}  (144)

This set of integers is generated by

n₁=0, min[m, n]

n ₂=0,m−n ₁

n ₃=0,n−n ₁   (145)

so

$\begin{matrix}\begin{matrix}{{K\left( {\lambda_{1},\lambda_{2},m,\lambda_{3},n} \right)} \equiv {\sum\limits_{K}\; {\frac{\lambda_{1}^{n_{1}}^{- \lambda_{1}}}{n_{1}!}\frac{\lambda_{2}^{n_{2}}^{- \lambda_{2}}}{n_{2}!}\frac{\lambda_{3}^{n_{3}}^{- \lambda_{3}}}{n_{3}!}}}} \\{= {\sum\limits_{{n_{1} = 0},{\min {\lbrack{m,n}\rbrack}}}\; {\frac{\lambda_{1}^{n_{1}}^{- \lambda_{1}}}{n_{1}!}{\sum\limits_{{n_{2} = 0},{m - n_{1}}}\; \frac{\lambda_{2}^{n_{2}}^{- \lambda_{2}}}{n_{2}!}}}}} \\{{\sum\limits_{n_{3} = {n - n_{1}}}\; \frac{\lambda_{3}^{n_{3}}^{- \lambda_{3}}}{n_{3}!}}} \\{= {\sum\limits_{{i = 0},{\min {\lbrack{m,n}\rbrack}}}\; {\frac{\lambda_{1}^{i}^{- \lambda_{1}}}{i!}{R\left( {\lambda_{2},{m - i}} \right)}{R\left( {\lambda_{3},{n - i}} \right)}}}}\end{matrix} & (146)\end{matrix}$

Notice that

K(λ₁, λ₂ , m, λ ₃ , n)=K(λ₁, λ₃ , n, λ ₂ , m)

K(λ₁, λ₂, 0, λ₃ , n)=^(−(λ) ¹ ^(+λ) ² ⁾ R(λ₃ , n)

K(λ₁, λ₂ , m, λ ₃, 0)=e ^(−(λ) ¹ ^(+λ) ³ ⁾ R(λ₂ , m)

K(λ₁, λ₂, 0, λ₃, 0)=e ^(−(λ) ¹ ^(+λ) ² ^(+λ) ³ ⁾

K(λ₁, λ₂, ∞, λ₃ , n)=e ^(−λ) ² R(λ₃ , n)

K(λ₁, λ₂ , m, λ ₃, ∞)=e ^(−λ) ² R(λ₂ , m)

K(λ₁, λ₂, ∞, λ₃, ∞)=1

K(λ₁, λ₂ , n, λ ₃ , m)=0 n<0 or m<0   (147)

The function K is easy to evaluate by computing the sum over products ofR in the last equation of Eq. (146) since m and n are small integers.

The present invention has now been described in accordance with severalexemplary embodiments, which are intended to be illustrative in allaspects, rather than restrictive. Thus, the present invention is capableof many variations in detailed implementation, which may be derived fromthe description contained herein by a person of ordinary skill in theart. For example:

-   -   The same system and method illustrated by application to the        design and manufacture of integrated circuits made of        subcircuits may be applied to the design and manufacture of        electronic systems made of components.    -   Models using non-exchangeable copulas of more than two        dimensions are covered by the methods of this invention.    -   Geometrical copulas of more than two dimensions may be        constructed using the same principles as the two-dimensional        geometrical copulas of the exemplary embodiments.    -   Dependency among attributes of different types, with different        marginal distributions and with different marginal environmental        dependence can be modeled using the methods of this invention.        This includes, for example, correlations among I_(sb), F_(max),        and product lifetime.    -   Modeled attributes of test vehicle and product may be in the        same Test or Use step or different Test or Use steps. For        example, a dependency of I_(sb) and F_(max) may be between        I_(sb) measured at Sort (wafer level test) and F_(max) measured        at Class (unit level Test), or may be between I_(sb) and F_(max)        both measured at Class.    -   The system and method covers use of all environmental conditions        which stress the unit while being tested or used. Temperature,        voltage and frequency are exemplary embodiments of environmental        conditions. Examples of other environmental conditions covered        by the method include delay settings, humidity, levels of        vibration, etc.    -   Products including multiple modules of several types rather than        the single type of module (bit) of the DRAM exemplary embodiment        are covered by the invention.    -   Models of multi-module and fault-tolerant products in which the        probability of occurrence of a defective module is distributed        according to the negative binomial distribution (a        generalization of the Poisson distribution) are covered by the        methods of the invention. Exemplary embodiments include        variation of defect density across wafers, lots and factories.    -   The modules, of which DRAM bits are an exemplary embodiment, can        be any element, which is characterized in a test vehicle and        used in one or more copies in a product. Other exemplary        embodiments include circuit blocks, and fuses in FPGA devices.    -   Computation of copula-dependent figures of merit other than the        ones shown in the exemplary embodiments is covered by the        methods of this invention.    -   The Bootstrap method was given in FIG. 32 b as a way to generate        Data Replicates that characterize sample variation. The        Bootstrap method is an exemplary embodiment of other data        resampling methods including Jackknife and Cross Validation,        which may be applied to characterize sample variation.    -   The principle of representation of key probabilities of product        Test/Use pass/fail categories by sums over constrained sets of        integers of counts of modules (e.g. bits) in various Test/Use        pass/fail categories applies to more complex array fault        tolerant schemes than were shown as exemplary embodiments.        Exemplary embodiments of more complex schemes are block-wise        replacements, and schemes in which spare bits are only available        within blocks but not globally.    -   A test vehicle may be an electronic system or integrated circuit        not intended for customer application, specifically designed to        facilitate data acquisition to build a statistical model. Or it        may be a product different from the product of interest, which        may be tested in a way that facilitates data acquisition to        build a statistical model.

All such variations are considered to be within the scope and spirit ofthe present invention as defined by the following claims and their legalequivalents.

What is claimed:
 1. A method implemented by an appropriately programmedcomputer for determining specifications that meet electronic system orintegrated circuit product requirements, comprising: a. acquiring datafrom a test vehicle; b. fitting said data to a copula-based statisticalmodel using an appropriately programmed computer; and c. using saidcopula-based statistical model to compute figures of merit using saidappropriately programmed computer, wherein said figures of merit areused to determine said specifications that meet said electronic systemor integrated circuit product requirements of an electronic system or aintegrated circuit product having correlated attributes.
 2. The methodof claim 1, wherein said data comprises values of attributes for eachmember of a population of said test vehicles manufactured by anelectronic system or integrated circuit manufacturing process measuredat specified environmental conditions.
 3. The method of claim 2, whereinsaid environmental conditions are selected from the group consisting oftemperature, voltage, and frequency.
 4. The method of claim 1, whereinsaid copula-based statistical model describes a dependency structure ofsaid data.
 5. The method of claim 1, wherein said copula-basedstatistical model comprises a copula and marginal distribution functionsthat describe a statistical distribution of each attribute of said data,wherein said copula and said marginal distribution functions embody adependency on environmental conditions.
 6. The method of claim 5,wherein said environmental conditions are selected from the groupconsisting of temperature, voltage, and frequency.
 7. The method ofclaim 5, wherein said copula is a geometrical copula, wherein saidgeometrical copula enables non-reject Monte-Carlo synthesis of syntheticdata used to compute said figures of merit.
 8. The method of claim 5,wherein said copula of said copula-based statistical model has taildependency structure characteristic to the physics of both said testvehicle and said product.
 9. The method of claim 5, wherein said copulais used to generate synthetic Monte-Carlo samples of instances of unitswith multiple attribute values, wherein said instances of unitscorrespond to a censored sample of a population of said product, andwherein attribute values are compared to said test specifications andsaid datasheet specifications to determine a pass or fail status,wherein said figures of merit are determined by counting instances ofsaid pass and fail status.
 10. The method of claim 9, wherein saidsynthetic Monte-Carlo samples are generated without rejection.
 11. Themethod of claim 9, wherein said copula is a geometric copula.
 12. Themethod of claim 1, wherein said specifications comprise design, test anddatasheet specifications.
 13. The method of claim 12 wherein designspecifications include representing the fault tolerance mechanisms ofthe product and the manufacturing test step as regions in a space givingallowed numbers of defective elements in various Test/Use pass/failcategories.
 14. The method of claim 12 wherein test specificationincludes representing the action of active repair at test as regions ina space giving allowed numbers of defective elements in various Test/Usepass/fail categories.
 15. The method of claim 1 further comprisesdetermining statistical confidence limits of said figures of merit,wherein said determination of statistical confidence limits of saidfigures of merit comprises computing distributions of said figures ofmerit from sets of replicates of model parameters generated fromresampled or synthesized test vehicle data using said appropriatelyprogrammed computer.
 16. The method of claim 15, wherein said resampledtest vehicle data reflecting variation in the design of the experimentthat produced the actual test vehicle data is generated by resamplingmethods, of which Bootstrap is an embodiment.
 17. The method of claim15, wherein said synthesized test vehicle data, comprises datareflecting experimental designs different from an experimental designthat produced actual test vehicle data is generated by using saidgeometrical copula to enable non-reject Monte-Carlo synthesis ofcensored test vehicle data.
 18. The method of claim 1, wherein saidfitting comprises fitting individual marginal attribute distributionmodels and said copula at a selected reference test coverage model,wherein said fitting of said individual marginal attribute distributionmodels and said copula may be done in any order.
 19. The method of claim1, wherein said acquisition of said data using said test vehiclecomprises measuring attributes separately on each sub-element on saidtest vehicle, wherein sub-elements of a product comprising a system orintegrated circuit is constructed.
 20. The method of claim 1, whereinsaid acquisition of said data using said test vehicle comprises a testprogram, wherein said test program disables all fault tolerancemechanisms in said test program and said test vehicle.
 21. The method ofclaim 1, wherein said acquisition of said data using said test vehiclecomprises an experimental design, wherein said experimental designcomprises conditions spanning possible datasheet specifications and testspecifications of a product.
 22. The method of claim 1 further comprisesdetermining whether said figures of merit of a new product satisfyquality, reliability, and cost requirements, wherein said new productcomprises design specifications, test specifications and datasheetspecifications that are different from design specifications and testspecifications of said test vehicle.
 23. The method of claim 22, whereinsaid different design specifications and different test specificationsinclude a different test coverage model from a reference test coveragemodel assumed in determining said statistical model from test vehicledata.
 24. The method of claim 22, wherein said different designspecifications include a number of circuit sub-elements or components insaid product that is different from the number of the same or similarcircuit sub-elements or components in said test vehicle.
 25. The methodof claim 22, wherein said different design specifications and saiddifferent test specifications comprise fault tolerance mechanisms and atest program used to acquire data from said test vehicle, wherein saidfault tolerance mechanisms are not enabled or not present in said testvehicle.
 26. The method of claim 1, wherein an analytical form of saidcopula-based statistical model is used by an appropriately programmedcomputer to enable deterministic (non-Monte-Carlo) calculation of saidfigures of merit to determine said design specifications, said testspecifications, and said datasheet specifications for said product.